Target image extraction methods, apparatus, devices and storage media
By extracting the connected components and centroid reconstruction of the target's mask image, and combining attention networks and convolution operations, the image deformation problem caused by support mechanism interference in optical property testing is solved, and high-precision target image extraction is achieved.
Patent Information
- Application Number
- CN202611328157.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
AI Technical Summary
In optical property testing scenarios, interference from the support mechanism causes distortion of the target image, and areas with gradual changes in brightness are misjudged as background, resulting in high target edge truncation errors and affecting image accuracy.
By extracting the connected components of the target's mask image, determining the centroid, reconstructing the mask image, obtaining the reconstructed image, and using the reconstructed image to extract images from the original and background images, performing pixel-by-pixel subtraction, and combining attention networks and convolution operations to generate the target image.
It reduces the impact of adhesion areas on the target image, improves the accuracy and realism of the image, reduces edge misjudgment, and improves image quality.
Smart Images

Figure CN123118682A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more specifically to a method, apparatus, device, and storage medium for extracting a target image. Background Technology
[0002] In optical property testing scenarios, a solar simulator is typically used to directionally illuminate the target, and a line-scan hyperspectral imaging system is used to acquire images of the target in order to obtain information such as the target's radiation characteristics based on these images.
[0003] In this optical property testing scenario, the target is usually placed on a support structure. During the acquisition of the target image, the target image is easily deformed due to interference from the support structure. Since the target is illuminated by simulated sunlight, there will be obvious areas of light and dark gradient on the target surface. Threshold segmentation method is generally used to process the image of the target after illumination. However, this method is prone to misjudging the dark diffuse reflection halo at the edge of the target as the background, causing the target boundary to shrink inward and producing extremely high edge truncation error. All of these will affect the accuracy of the final target image. Summary of the Invention
[0004] In view of the above problems, embodiments of this application provide a target image extraction method, apparatus, device, and storage medium.
[0005] According to a first aspect of this application, a target image extraction method is provided, comprising: extracting connected components of a mask image of a target, and taking the connected component with the largest area as the target connected component; wherein the mask image is obtained based on an original image containing the target and a background image not containing the target in the same region; determining the centroid of the target connected component, and reconstructing the mask image based on the centroid to obtain a reconstructed image; wherein the region corresponding to the reconstructed image includes the region in the region where the target is located, excluding the adhesion region, and the adhesion region represents the region in the original image where the target is connected to a non-target; extracting a first image from the original image and a second image from the background image using the reconstructed image; and subtracting the first image and the second image pixel by pixel to obtain the target image.
[0006] According to embodiments of this application, extracting a first image from an original image and a second image from a background image using a reconstructed image includes: obtaining an image region corresponding to a target pixel in the reconstructed image; wherein the pixel value of the target pixel is greater than the pixel values of other pixels in the reconstructed image; using the image corresponding to the image region in the original image as the first image; and using the image corresponding to the image region in the background image as the second image.
[0007] According to embodiments of this application, reconstructing a mask image based on the centroid and obtaining the reconstructed image includes: inputting the position coordinates of the centroid into a position embedding encoder to obtain a position encoding vector; extracting image features from the mask image to obtain an image feature vector; using the position encoding vector as a query vector and the image feature vector as a key vector and a value vector, calculating the correlation between the image features and the centroid through an attention network, and weighting the value vector according to the correlation to obtain an attention feature vector; fusing the attention feature vector and the image feature vector to obtain a fused vector and performing a convolution operation to obtain a mask probability map; and binarizing the mask probability map to generate the reconstructed image.
[0008] According to an embodiment of this application, determining the centroid of a target connected region includes: determining the circumscribed boundary values of the target connected region in each direction, and determining the length of the target connected region based on the difference between a first circumscribed boundary value and a second circumscribed boundary value in a first direction; wherein the first direction represents the length direction of the target connected region, the first circumscribed boundary value represents the maximum circumscribed boundary value of the target connected region in the first direction, and the second circumscribed boundary value represents the minimum circumscribed boundary value of the target connected region in the first direction; when the length and width of the target connected region are the same, determining the centroid based on a third circumscribed boundary value and the length of the target connected region in a second direction; wherein the second direction represents the width direction of the target connected region, and the third circumscribed boundary value represents the boundary value of the circumscribed boundary away from the adhesion region in the second direction.
[0009] According to an embodiment of this application, before extracting the connected components of the mask image of the target, the method includes: dilating the region where the target is located in the original image to obtain a dilated region; extracting a third image from the background image based on the dilated region; calculating the mean pixel value and the standard deviation pixel value of the third image; and calculating a threshold using the mean pixel value and the standard deviation pixel value according to a preset strategy.
[0010] According to an embodiment of this application, extracting a third image from a background image based on an inflated region includes: subtracting the background image pixel by pixel from the image corresponding to the inflated region in the background image to obtain the third image.
[0011] According to an embodiment of this application, the mask image is obtained by subtracting the original image from the background image pixel by pixel to obtain a difference image; determining the first pixel corresponding to the pixel value greater than a threshold in the difference image, and performing binarization processing and edge repair operation on the image composed of the first pixel to obtain the mask image.
[0012] According to a second aspect of this application, a target image extraction apparatus is provided, comprising: a target connected component acquisition module, configured to extract connected components of a mask image of a target, and to select the connected component with the largest area as the target connected component; wherein the mask image is obtained based on an original image containing the target and a background image not containing the target within the same region; a reconstructed image acquisition module, configured to determine the centroid of the target connected component, and to reconstruct the mask image based on the centroid to obtain a reconstructed image; wherein the region corresponding to the reconstructed image includes the region in the region where the target is located, excluding the adhesion region, and the adhesion region represents the region in the original image where the target is connected to a non-target; an image extraction module, configured to extract a first image from the original image and a second image from the background image using the reconstructed image; and a target image acquisition module, configured to subtract the first image from the second image pixel by pixel to obtain the target image.
[0013] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.
[0014] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method. Attached Figure Description
[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:
[0016] Figure 1 The illustration schematically depicts an application scenario of the target image extraction method, apparatus, device, and storage medium according to embodiments of this application;
[0017] Figure 2 A flowchart illustrating a target image extraction method according to an embodiment of this application is shown schematically.
[0018] Figure 3 This illustration schematically shows a method for obtaining the centroid coordinates of a target connected domain according to an embodiment of this application.
[0019] Figure 4 A schematic diagram illustrating a reconstructed image acquisition method according to an embodiment of this application is shown.
[0020] Figure 5 The illustration shows a comparison between target images obtained using the target image extraction method and the threshold segmentation method according to embodiments of this application;
[0021] Figure 6This schematic diagram illustrates a structural block diagram of a target image extraction apparatus according to an embodiment of the present application;
[0022] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a target image extraction method according to an embodiment of this application. Detailed Implementation
[0023] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.
[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.
[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.
[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).
[0027] Figure 1 The illustration depicts an application scenario of the target image extraction method according to an embodiment of this application. For example... Figure 1As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.
[0028] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.
[0029] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and basic cloud computing services such as big data. The server can be the backend server of the aforementioned target application, used to provide backend services to the clients of the target application.
[0030] It should be noted that the target image extraction method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-103. Accordingly, the target image extraction device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-103.
[0031] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0032] Figure 2 A flowchart illustrating a target image extraction method according to an embodiment of this application is shown schematically. Figure 2 As shown, the target image extraction method 200 according to the embodiments of this application may include steps S210 to S240.
[0033] In step S210, the connected components of the target's mask image are extracted, and the connected component with the largest area is taken as the target connected component; wherein, the mask image is obtained based on the original image containing the target and the background image not containing the target in the same region.
[0034] In step S220, the centroid of the target connected region is determined, and the mask image is reconstructed based on the centroid to obtain the reconstructed image; wherein, the region corresponding to the reconstructed image includes the region in the target area excluding the adhesion region, and the adhesion region represents the region in the original image where the target and non-target are connected.
[0035] In step S230, a first image is extracted from the original image using the reconstructed image, and a second image is extracted from the background image.
[0036] In step S240, the first image and the second image are subtracted pixel by pixel to obtain the target image.
[0037] In some embodiments, in step S210, the target and the supporting structure supporting the target are irradiated using a solar simulator, and an original image and a background image are acquired using a line-scan hyperspectral imaging system. The original image refers to the image obtained after imaging the background and target, including the supporting structure, shadows, etc., after irradiation. The original image includes the target and the background, and the background includes the supporting structure, shadows, etc. The background image refers to the image obtained after imaging the background that has not been irradiated by the solar simulator. The background image only includes the background and does not include the target. The irradiated area corresponding to the original image is the same as the irradiated area corresponding to the background image.
[0038] Furthermore, a line-scan hyperspectral imaging system is used to acquire spectral cube data of the target and supporting structures after illumination by a solar simulator. Spectral cube data is three-dimensional data, including a first dimension, a second dimension, and a third dimension. The first dimension can refer to the spatial width, the second dimension to the spatial length, and the third dimension to the spectral band dimension. In other words, spectral cube data can include multiple bands. Assuming the spectral cube data includes three bands, the average gray values of the spectral cube data within the first band, the average gray values within the second band, and the average gray values within the third band are calculated. These average gray values are then fused to obtain a wideband grayscale image corresponding to the spectral cube data. This wideband grayscale image is the original image, used... This refers to the original image.
[0039] A line-scan hyperspectral imaging system was used to acquire spectral cube data of background structures, such as support structures, that were not illuminated by the solar simulator. Following the method used to acquire the original image, the average grayscale values of the background spectral cube data in each band were fused to obtain a background image. This represents the background image.
[0040] In some embodiments, in step S220, the non-target refers to the support mechanism that supports the target.
[0041] According to embodiments of this application, by reconstructing the target mask image through the centroid of the target connected region, the influence of non-targets in the adhesion region on the target mask image is greatly reduced, improving the accuracy and realism of the target image. By reconstructing the image, a first image is extracted from the original image, a second image is extracted from the background image, and the image obtained by subtracting the second image from the first image is used as the target image. This allows for the accurate extraction of the target corresponding region images from the original image and the background image, thereby obtaining the target image, reducing misjudgments caused by target edges, and improving the accuracy of the target image.
[0042] In some embodiments, before extracting the connected components of the mask image of the target, the method includes: dilating the region where the target is located in the original image to obtain a dilated region; extracting a third image from the background image based on the dilated region; calculating the mean and standard deviation of the pixel values of the third image; and calculating a threshold using the mean and standard deviation of the pixel values according to a preset strategy.
[0043] Furthermore, since the area where the target is located is a bright reflective area, a square frame larger than the area where the target is located can be used to scan the bright reflective area. For example, a square frame of 40 pixels × 40 pixels can be used to scan the bright reflective area and dilate the bright reflective area to obtain the expanded area.
[0044] The third image is a cleaner background image; in other words, the noise in the third image is less than the noise in the background image. The average pixel value of the third image is... The standard deviation of pixel values is Then the threshold The calculation method is as follows:
[0045]
[0046] This threshold calculation method accurately captures the edge of the extremely dark gradient halo of the target under simulated sunlight, and avoids cutting off the dark edge through pure geometric reconstruction.
[0047] According to embodiments of this application, dilating the region containing the target in the original image helps to obtain a cleaner background image, which further filters out noise in the background image, thereby reducing noise in the target image and improving image quality. Determining the threshold using the mean and standard deviation of pixel values in the third image allows the threshold to be dynamically adjusted according to the pixel features of the third image. Since the third image is extracted from the background image, the threshold can also be dynamically adjusted according to the pixel features of the background image, thus improving the reliability of the threshold.
[0048] In some embodiments, extracting a third image from a background image based on the dilated region includes: subtracting the background image pixel by pixel from the image corresponding to the dilated region in the background image to obtain the third image.
[0049] Furthermore, the background image is subtracted pixel by pixel from the image corresponding to the dilated region in the background image. In other words, the image of the region that is the same as the position range indicated by the dilated region is removed from the background image.
[0050] According to the embodiments of this application, removing the image corresponding to the dilated region from the background image can yield a cleaner background image with less noise, i.e., a third image. This process suppresses noise while preserving the effective information of the background image, thereby enhancing the reliability of image processing.
[0051] In some embodiments, the mask image is obtained by subtracting the original image from the background image pixel by pixel to obtain a difference image; determining the first pixel corresponding to the pixel value greater than a threshold in the difference image, and performing binarization and edge repair operations on the image composed of the first pixel to obtain the mask image.
[0052] Furthermore, the pixel difference between the original image and the background image can be set to 0 if it is a negative number to obtain a difference image, thereby eliminating shadow interference. The pixel difference between the original image and the background image is the pixel value at the corresponding position in the difference image.
[0053] Since the image composed of the first pixel includes the target edge, binarizing the image composed of the first pixel in the difference image can obtain an initial mask image that encloses the dark field radiation boundary of the target. Then, a slight morphological closing operation is used to repair the broken edges in the initial mask image to obtain the target mask image, which contains all the information of the target. This method can improve image quality.
[0054] According to embodiments of this application, by performing binarization and edge repair operations on the image composed of pixels larger than a threshold in the differential image, the mask image of the target region can be initially extracted, and the broken target edges in the mask image can be repaired, thereby improving the accuracy and image quality of the mask image.
[0055] Figure 3 The illustration shows a schematic diagram of a method for obtaining the centroid position coordinates of a target connected region according to an embodiment of this application.
[0056] In some embodiments, determining the centroid of a target connected region includes: determining the circumscribed boundary values of the target connected region in each direction, and determining the length of the target connected region based on the difference between a first circumscribed boundary value and a second circumscribed boundary value in a first direction; wherein the first direction represents the length direction of the target connected region, the first circumscribed boundary value represents the maximum circumscribed boundary value of the target connected region in the first direction, and the second circumscribed boundary value represents the minimum circumscribed boundary value of the target connected region in the first direction; when the length and width of the target connected region are the same, determining the centroid based on a third circumscribed boundary value and the length of the target connected region in a second direction; wherein the second direction represents the width direction of the target connected region, and the third circumscribed boundary value represents the boundary value of the circumscribed boundary away from the adhesion region in the second direction.
[0057] The target connected region includes the extremely dark halo region of the target; that is, the target connected region includes the adhered region. The circumscribed boundary values of the target connected region in each direction are obtained through the circumscribed rectangle (i.e., the minimum circumscribed boundary rectangle), and the centroid is determined based on these circumscribed boundary values. Since the adhered region affects the accuracy of the centroid's position, the circumscribed boundary values at the adhered region are not considered. Specifically, assuming the adhered region is located at the bottom of the target, the length of the target connected region can be determined based on the rightmost first circumscribed boundary value and the leftmost second circumscribed boundary value. The position corresponding to half of this length is the centroid's position in the first direction. The position of the centroid in the second direction can be determined based on the width of the target connected region and the topmost third circumscribed boundary value; that is, the position of the centroid in the second direction is the location of the third circumscribed boundary at half the width of the target connected region. Based on the centroid's positions in the first and second directions, the position coordinates of the centroid are obtained.
[0058] If the target connected region is a regular area, such as a cube or a sphere, then after determining the length of the target connected region, assuming the length and width of the target connected region are the same, the position of the centroid in the second direction can be obtained based on the length of the target connected region and the value of the third circumscribed boundary, thus obtaining the position coordinates of the centroid. The calculation method of the centroid is illustrated using a support mechanism where the target is a sphere and the non-target is a sphere as an example: Figure 3 As shown, the first circumscribed boundary value is The second circumscribed boundary value is The third circumscribed boundary value is The true radius of the sphere for:
[0059] Establish a coordinate system by setting the x-axis from left to right and the y-axis from top to bottom. Then the coordinates of the centroid in the first direction (horizontal direction) are... for: The coordinates of the centroid in the second direction (vertical direction) for: Thus, the position coordinates of the centroid are obtained. Furthermore, the fourth circumscribed boundary value of the target connected component in the second direction. The corresponding circumscribed boundary is located in the adhesion region. To ensure the accuracy of the centroid position of the target connected domain, the fourth circumscribed boundary value is not used to calculate the position coordinates of the centroid.
[0060] According to the embodiments of this application, when the length and width of the target connected region are the same, the centroid position of the target connected region can be determined based on the first circumscribed boundary value, the second circumscribed boundary value, and the third circumscribed boundary value. This greatly reduces problems such as distortion of the reconstructed mask image caused by the adhesion between the non-target and the target in the adhesion region, and improves the accuracy of the reconstructed image.
[0061] Figure 4 The illustration shows a schematic diagram of a reconstructed image acquisition method according to an embodiment of this application.
[0062] like Figure 4As shown, based on the centroid, the mask image is reconstructed to obtain the reconstructed image, including: inputting the centroid's position coordinates 401 into the position embedding encoder 402 to obtain the position encoding vector 403; extracting image features from the mask image 404 to obtain the image feature vector 405; using the position encoding vector as the query vector and the image feature vector as the key vector and value vector, calculating the correlation between the image features and the centroid through an attention network, and weighting the value vector according to the correlation to obtain the attention feature vector 406; fusing the attention feature vector and the image feature vector to obtain the fusion vector 407; executing step S410 to perform a convolution operation to generate the reconstructed image, that is, performing a convolution operation on the fusion vector 407 to obtain the mask probability map; and binarizing the mask probability map to generate the reconstructed image.
[0063] Furthermore, image features include texture features, color features, and shape features. The correlation between image features and the centroid refers to the distance between the region corresponding to the image feature and the centroid. For example, texture features around the centroid have a high correlation with the centroid because the region corresponding to the texture feature is located around the centroid. This indicates that the texture feature is not a texture feature of the image near the adhesion region, and can be preserved to reconstruct the mask image. If the distance between the region corresponding to a certain image block and the centroid is greater than a distance threshold, it indicates that the region corresponding to that image block is likely an adhesion region, and the image features of that image block cannot be preserved. The distance threshold can be determined based on the length and width of the target connected components.
[0064] Since the position encoding vector is a one-dimensional vector and the image feature vector is a multi-dimensional vector, a cross-attention operation is performed using the position encoding vector as the query vector and the image feature vector as the key and value vectors. The resulting attention feature vector is a one-dimensional vector. To enrich the details of the reconstructed image, it is necessary to convolve the multi-dimensional vector. Therefore, the attention feature vector and the image feature vector are fused to obtain multiple-dimensional image features that match the attention feature vector from the image feature vector. This allows the local details of the mask image to be preserved while retaining the centroid position information, thereby improving the accuracy of the reconstructed image.
[0065] Furthermore, based on the characteristics of the reconstructed image, evaluation metrics can be quantified to assess the quality of the target image. For example, spectral features are extracted from the original spectral cube data based on the coordinates of each pixel in the reconstructed image. Then, evaluation metrics are calculated based on the photon energy at the corresponding positions indicated by the spectral features. The original spectral cube data refers to the spectral cube data of the target and its supporting structures, acquired using a line-scan hyperspectral imaging system after illumination by a solar simulator. Evaluation metrics include edge roundness index, net energy recovery rate, and edge truncation error. Edge roundness index... It can be calculated using the following formula:
[0066]
[0067] Represents the pixel area of the reconstructed image. Represents the perimeter of the reconstructed image boundary. Net energy recovery rate. The calculation method is as follows:
[0068] , These represent the energy of photons received by pixels within the reconstructed image and the energy of photons received by pixels at the edge after the reconstructed image is expanded outwards, respectively. Edge truncation error. for:
[0069]
[0070] Furthermore, , , Indicates that it is located at The energy of the photons received by the pixel at that location. Indicates the reconstruction of the image interior. This represents the edge region after the reconstructed image is expanded outwards.
[0071] Compared with the method of extracting target images using threshold segmentation, the target image extraction method of this application improves the edge roundness index of the reconstructed image from 0.7895 to 0.8932, solves the geometric distortion caused by the adhesion of the support mechanism, and almost perfectly restores the projected cross section of the target. The edge truncation error is less than 1%, and the net energy recovery rate is as high as 99% or more, which greatly improves the accuracy of optical property testing.
[0072] According to embodiments of this application, an attention network is used to calculate the correlation between image features and the centroid, so as to retain image features in areas close to the centroid. This not only reduces the impact of target edges on the target image, but also preserves the detailed information of the mask image, thereby improving the quality of the target image. By incorporating the positional information of the centroid into the image features and preserving the local detailed information of the mask image, a fusion vector that combines positional information and image detail information is obtained, which is beneficial to improving the accuracy of the reconstructed image.
[0073] In some embodiments, extracting a first image from an original image and a second image from a background image using a reconstructed image includes: obtaining an image region corresponding to a target pixel in the reconstructed image; wherein the pixel value of the target pixel is greater than the pixel values of other pixels in the reconstructed image; using the image corresponding to the image region in the original image as the first image; and using the image corresponding to the image region in the background image as the second image.
[0074] Since the reconstructed image is a binary image, assuming the target pixel has a pixel value of 1, we determine the image region with a pixel value of 1 in the reconstructed image. This image region does not contain any adhered regions and can accurately, realistically, and completely represent the structure and shape of the target. We extract the first image corresponding to this image region from the original image and the second image corresponding to this image region from the background image. Alternatively, we can extract the second image corresponding to this image region from a cleaner background image (the third image). Then, we subtract the first image from the second image pixel by pixel to obtain the target image.
[0075] According to embodiments of this application, based on the image region corresponding to the target pixel in the reconstructed mask image, a first image is extracted from the original image and a second image is extracted from the background image. This can further remove the influence of the target edge, ensuring that neither the first image nor the second image contains the target edge, thereby improving the accuracy of the target image.
[0076] Figure 5 The illustration shows a comparison between target images obtained using the target image extraction method and the target image obtained using the threshold segmentation method according to embodiments of this application.
[0077] like Figure 5 As shown, Figure (a) represents the initial image containing the target after being illuminated by the solar simulator, Figure (b) represents the target image obtained using the threshold segmentation method, and Figure (c) represents the target image obtained using the target image extraction method of this application. The horizontal coordinates of these three images represent the horizontal coordinate positions of the pixels, and the vertical coordinates represent the vertical coordinate positions of the pixels. By comparing these three images, it can be found that the target image obtained using the target image extraction method of this application is clearer, has smoother edge transitions, and has better image quality.
[0078] Based on the above-described target image extraction method, embodiments of this application also provide a target image extraction apparatus. The following will be combined with... Figure 6 The device is described in detail.
[0079] Figure 6 A schematic block diagram of a target image extraction apparatus according to an embodiment of this application is shown.
[0080] like Figure 6 As shown, the target image extraction device 600 of this embodiment includes a target connected component acquisition module 610, a reconstructed image acquisition module 620, an image extraction module 630, and a target image acquisition module 640.
[0081] The target connected component acquisition module 610 is used to extract connected components from the mask image of the target, and select the connected component with the largest area as the target connected component; wherein, the mask image is obtained based on the original image containing the target and the background image not containing the target in the same region. In one embodiment, the target connected component acquisition module 610 can be used to perform step S210 described above, which will not be repeated here.
[0082] The reconstructed image acquisition module 620 is used to determine the centroid of the target connected region, and reconstruct the mask image based on the centroid to obtain the reconstructed image. The region corresponding to the reconstructed image includes the region where the target is located, excluding the adhesion region. The adhesion region represents the region in the original image where the target is connected to a non-target. In one embodiment, the reconstructed image acquisition module 620 can be used to execute step S220 described above, which will not be repeated here.
[0083] Image extraction module 630 is used to extract a first image from the original image using the reconstructed image, and to extract a second image from the background image. In one embodiment, image extraction module 630 can be used to perform step S230 described above, which will not be repeated here.
[0084] The target image acquisition module 640 is used to subtract the first image from the second image pixel by pixel to obtain the target image. In one embodiment, the target image acquisition module 640 can be used to perform step S240 described above, which will not be repeated here.
[0085] According to the embodiments of this application, the target image extraction device 600 can reduce the influence of non-targets in the adhesion region on the mask image of the target, thereby improving the accuracy and authenticity of the target image. By reconstructing the image, extracting the first image from the original image and the second image from the background image, and subtracting the second image from the first image to obtain the target image, the image corresponding to the target can be accurately extracted from the original image and the background image respectively, thereby obtaining the target image, reducing misjudgment caused by the target edge, and improving the accuracy of the target image.
[0086] In some embodiments, the image extraction module 630 is specifically used to: obtain the image region corresponding to the target pixel in the reconstructed image; wherein the pixel value of the target pixel is greater than the pixel value of other pixels in the reconstructed image; take the image corresponding to the image region in the original image as the first image; and take the image corresponding to the image region in the background image as the second image.
[0087] In some embodiments, the reconstructed image acquisition module 620 is specifically used for: inputting the centroid's position coordinates into a position embedding encoder to obtain a position encoding vector; extracting image features from a mask image to obtain an image feature vector; using the position encoding vector as a query vector and the image feature vector as a key vector and value vector, calculating the correlation between the image features and the centroid through an attention network, and weighting the value vector according to the correlation to obtain an attention feature vector; fusing the attention feature vector and the image feature vector to obtain a fused vector and performing a convolution operation to obtain a mask probability map; and binarizing the mask probability map to generate a reconstructed image.
[0088] In some embodiments, the reconstructed image acquisition module 620 is further configured to: determine the circumscribed boundary values of the target connected region in each direction, and determine the length of the target connected region based on the difference between the first circumscribed boundary value and the second circumscribed boundary value in the first direction; wherein the first direction represents the length direction of the target connected region, the first circumscribed boundary value represents the maximum circumscribed boundary value of the target connected region in the first direction, and the second circumscribed boundary value represents the minimum circumscribed boundary value of the target connected region in the first direction; and when the length and width of the target connected region are the same, determine the centroid based on the third circumscribed boundary value and the length of the target connected region in the second direction; wherein the second direction represents the width direction of the target connected region, and the third circumscribed boundary value represents the boundary value of the circumscribed boundary away from the adhesion region in the second direction.
[0089] In some embodiments, the target image extraction device 600 is further configured to: dilate the region where the target is located in the original image to obtain a dilated region; extract a third image from the background image based on the dilated region; calculate the mean pixel value and standard deviation of the pixel value of the third image; and calculate a threshold using the mean pixel value and standard deviation of the pixel value according to a preset strategy.
[0090] In some embodiments, the target image extraction device 600 is further configured to: subtract the background image from the image corresponding to the dilated region in the background image pixel by pixel to obtain a third image.
[0091] In some embodiments, the target image extraction device 600 is further configured to: subtract the original image from the background image pixel by pixel to obtain a difference image; determine the first pixel corresponding to the pixel value greater than a threshold in the difference image; and perform binarization processing and edge repair operation on the image composed of the first pixel to obtain a mask image.
[0092] According to embodiments of this application, any multiple modules among the target connected component acquisition module 610, reconstructed image acquisition module 620, image extraction module 630, and target image acquisition module 640 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the target connected component acquisition module 610, reconstructed image acquisition module 620, image extraction module 630, and target image acquisition module 640 can be at least partially implemented as hardware circuitry, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the target connected component acquisition module 610, the reconstructed image acquisition module 620, the image extraction module 630, and the target image acquisition module 640 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.
[0093] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a target image extraction method according to an embodiment of this application.
[0094] like Figure 7 As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory 702 or a program loaded from a storage portion 708 into a random access memory 703. The processor 701 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.
[0095] Random access memory 703 stores various programs and data required for the operation of electronic device 700. Processor 701, read-only memory 702, and random access memory 703 are interconnected via bus 704. Processor 701 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 702 and / or random access memory 703. It should be noted that the programs may also be stored in one or more memories other than read-only memory 702 and random access memory 703. Processor 701 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.
[0096] According to embodiments of this application, the electronic device 700 may further include an input / output interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card, such as a local area network card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.
[0097] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.
[0098] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 702, and / or random access memory 703, and / or one or more memories other than read-only memory 702 and random access memory 703 described above.
[0099] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.
[0100] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.
[0101] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.
[0102] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0104] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.
Claims
1. A method for extracting a target image, characterized in that, The method includes: The connected components of the target's mask image are extracted, and the connected component with the largest area is taken as the target connected component; wherein, the mask image is obtained based on the original image containing the target and the background image not containing the target in the same region; Determine the centroid of the target connected component, and reconstruct the mask image based on the centroid to obtain the reconstructed image; wherein, the region corresponding to the reconstructed image includes the region in the target area excluding the adhesion region, and the adhesion region represents the region in the original image where the target and non-target are connected; The reconstructed image is used to extract a first image from the original image and a second image from the background image; The target image is obtained by subtracting the first image from the second image pixel by pixel.
2. The method according to claim 1, characterized in that, The step of extracting a first image from the original image and a second image from the background image using the reconstructed image includes: Obtain the image region corresponding to the target pixel in the reconstructed image; wherein the pixel value of the target pixel is greater than the pixel values of other pixels in the reconstructed image; The image corresponding to the image region in the original image is used as the first image; and The image corresponding to the image region in the background image is used as the second image.
3. The method according to claim 1, characterized in that, The step of reconstructing the mask image based on the centroid to obtain the reconstructed image includes: The position coordinates of the centroid are input into the position embedding encoder to obtain the position encoding vector. Image features are extracted from the mask image to obtain the image feature vector. Using the location encoding vector as the query vector and the image feature vector as the key vector and value vector, the degree of correlation between the image features and the centroid is calculated through an attention network, and the value vector is weighted according to the degree of correlation to obtain the attention feature vector; The attention feature vector and the image feature vector are fused to obtain a fused vector, and then a convolution operation is performed to obtain a mask probability map. The mask probability map is binarized to generate the reconstructed image.
4. The method according to claim 1, characterized in that, Determining the centroid of the target connected component includes: The outer boundary values of the target connected component in each direction are determined, and the length of the target connected component is determined based on the difference between the first outer boundary value and the second outer boundary value in the first direction; wherein, the first direction represents the length direction of the target connected component, the first outer boundary value represents the maximum outer boundary value of the target connected component in the first direction, and the second outer boundary value represents the minimum outer boundary value of the target connected component in the first direction; When the length and width of the target connected region are the same, the centroid is determined based on the third circumscribed boundary value of the target connected region in the second direction and the length; wherein, the second direction represents the width direction of the target connected region, and the third circumscribed boundary value represents the boundary value of the circumscribed boundary away from the adhesion region in the second direction.
5. The method according to claim 1, characterized in that, Before extracting the connected components of the mask image of the target, the following is included: The region where the target is located in the original image is dilated to obtain a dilated region. Based on the dilated region, a third image is extracted from the background image. Calculate the mean and standard deviation of the pixel values in the third image; According to a preset strategy, the threshold is calculated using the mean pixel value and the standard deviation of the pixel value.
6. The method according to claim 5, characterized in that, The step of extracting a third image from the background image based on the dilated region includes: The third image is obtained by subtracting the background image from the image corresponding to the dilated region in the background image pixel by pixel.
7. The method according to claim 5, characterized in that, The mask image is obtained in the following way: The original image and the background image are subtracted pixel by pixel to obtain a difference image; The first pixel corresponding to the pixel value greater than the threshold in the difference image is determined, and the image composed of the first pixel is binarized and edge repaired to obtain the mask image.
8. A target image extraction device, characterized in that, The device includes: The target connected component acquisition module is used to extract the connected components of the mask image of the target, and take the connected component with the largest area as the target connected component; wherein, the mask image is obtained based on the original image containing the target and the background image not containing the target in the same region; The reconstructed image acquisition module is used to determine the centroid of the target connected region, and reconstruct the mask image based on the centroid to obtain the reconstructed image; wherein, the region corresponding to the reconstructed image includes the region in the target area excluding the adhesion region, and the adhesion region represents the region in the original image where the target and non-target are connected; An image extraction module is used to extract a first image from the original image and a second image from the background image using the reconstructed image; The target image acquisition module is used to subtract the first image from the second image pixel by pixel to obtain the target image.
9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.
