Image region of interest determination method and device, electronic equipment and storage medium
By inputting test images into a segmentation model for classification and object detection, and locking in regions of interest based on test metrics, the problem of incorrect region of interest selection in existing technologies is solved, thus improving the accuracy and consistency of image analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI WINGTECH INFORMATION TECH CO LTD
- Filing Date
- 2023-03-09
- Publication Date
- 2026-05-15
AI Technical Summary
Existing methods for determining regions of interest often result in selection errors in image analysis, especially during subjective debugging, leading to inaccurate test results.
By acquiring test images and inputting them into a pre-trained segmentation model for classification, and using convolutional neural networks for object detection and semantic segmentation, the labeled regions of the target object categories are identified as regions of interest based on test metrics.
It improves the accuracy of region of interest determination, ensures the accuracy and consistency of image testing, and reduces misclassification and noise testing errors, especially in subjective debugging.
Smart Images
Figure CN116310298B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to a method, apparatus, electronic device, and storage medium for determining the region of interest in an image. Background Technology
[0002] A Region of Interest (ROI) is a region of image selected from an image. This region is the focus of image analysis testing and usually needs to be delineated for further processing.
[0003] With automated debugging gaining traction, the selection of Regions of Interest (ROIs) is crucial, whether for objective or subjective debugging. Image analysis and processing are performed according to the selected ROI; the more precise and accurate the chosen ROI, the closer the test results will be to the ideal outcome. Existing ROI determination methods typically lock onto corresponding regions based on desired features, but these methods often result in incorrect ROI selection. Therefore, improving the accuracy of determining ROIs has become a pressing issue. Summary of the Invention
[0004] This application discloses a method, apparatus, electronic device, and storage medium for determining the region of interest (ROI) of an image, which can improve the accuracy of determining the ROI of an image.
[0005] This application discloses a method for determining the region of interest in an image, characterized in that the method includes:
[0006] Acquire the first test image captured by the test equipment;
[0007] The first test image is input into a trained segmentation model for classification to obtain a second test image; the second test image includes multiple labeled regions corresponding to different object categories.
[0008] Based on the target object category corresponding to the test index of the first test image, a target labeled region matching the target object category is determined from the second test image, which is then used as the region of interest of the first test image.
[0009] As an optional implementation, before determining a target annotation region in the second test image that matches the target object category based on the test index of the first test image, and using this region as the region of interest of the first test image, the method further includes:
[0010] Based on the mapping relationship between test indicators and object categories, the target object category corresponding to the test indicator of the first test image is determined; the mapping relationship between the test indicator and the object category is determined based on the degree to which the image features of the object reflect the test indicator.
[0011] As an optional implementation, the second test image is obtained by the segmentation model color-coding the regions corresponding to each object in the first test image based on the category to which each object belongs in the first test image and a preset mapping relationship between category and color.
[0012] As an optional implementation, after determining a target annotation region in the second test image that matches the target object category based on the test index of the first test image, and using this region as the region of interest of the first test image, the method further includes:
[0013] Acquire a first standard image captured by the comparison device;
[0014] Based on the first standard image, the region of interest of the first test image is tested.
[0015] As an optional implementation, the step of testing the region of interest of the first test image based on the first standard image includes:
[0016] The first standard image is input into the segmentation model to obtain a second standard image, which includes multiple standard labeled regions with color.
[0017] The region of interest of the first standard image is determined from multiple labeled regions of the second standard image;
[0018] Based on the region of interest of the first standard image, the region of interest of the first test image is tested.
[0019] As an optional implementation, the annotation color corresponding to the region of interest in the first standard image is the same as the annotation color corresponding to the region of interest in the first test image.
[0020] As an optional implementation, the method for generating the segmentation model includes:
[0021] Acquire multiple sample images; the multiple sample images include multiple images acquired based on different shooting parameters;
[0022] Based on the multiple sample images, multiple labeled images are obtained; the multiple labeled images are obtained by labeling the multiple sample images based on the user's classification requirements; each labeled image includes multiple labeled regions annotated by the user;
[0023] The segmentation model is trained based on the multiple labeled images.
[0024] This application discloses an apparatus for determining the region of interest in an image, the apparatus comprising:
[0025] The acquisition module is used to acquire the first test image captured by the test equipment;
[0026] The input module is used to input the first test image into a pre-trained segmentation model for classification to obtain a second test image; the second test image includes multiple labeled regions corresponding to different object categories;
[0027] The determination module is used to determine, based on the target object category corresponding to the test index of the first test image, a target annotation region matching the target object category in the second test image, as the region of interest of the first test image.
[0028] This application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor enables the processor to implement any of the image region of interest determination methods disclosed in this application.
[0029] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any of the image region of interest determination methods disclosed in this application.
[0030] Compared with related technologies, the embodiments of this application have the following beneficial effects:
[0031] A first test image captured by a testing device is input into a pre-trained segmentation model for classification, resulting in a second test image. The second test image includes multiple labeled regions corresponding to different object categories. Based on the target object category corresponding to the test index of the first test image, a target labeled region matching the target object category is determined from the multiple labeled regions, serving as the region of interest (ROI) of the first test image. This embodiment first segments the first test image into labeled regions corresponding to different object categories using a segmentation model. Based on the target object category corresponding to the test index of the first test image, it accurately locks the labeled region matching the target object category from the multiple labeled regions corresponding to different object categories as the ROI, thus improving the accuracy of determining the ROI of the image. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a flowchart illustrating a method for determining the region of interest in an image, as disclosed in an embodiment of this application.
[0034] Figure 2 This is a flowchart illustrating another method for determining the region of interest in an image, as disclosed in an embodiment of this application.
[0035] Figure 3 This is a user interface diagram of a method for determining the region of interest in an image disclosed in an embodiment of this application;
[0036] Figure 4 This is a flowchart illustrating another method for determining the region of interest in an image, as disclosed in an embodiment of this application.
[0037] Figure 5 This is a schematic diagram of the structure of an image region of interest determination device disclosed in an embodiment of this application;
[0038] Figure 6 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation
[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0040] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0041] In related technologies, methods for determining regions of interest (ROIs) can be applied to objective image processing. Objective processing involves fixed content and area size, with controllable variables. The content may include fixed image cards, with similar card positions. The ROI selected for each processing iteration is generally consistent. For example, during shooting, the image card is centered, occupying approximately 75%-80% of the frame. The center image requires the card to be positioned at a 4:2 ratio at the four corners. Based on this content, the location of the ROI is generally known, making it relatively easy to select the correct ROI for objective testing. However, objective testing demands high-level shooting techniques, especially with wide-angle lenses. Even slight deviations in shooting can prevent the selection of the desired ROI, necessitating reshoots.
[0042] In related technologies, methods for determining regions of interest (ROIs) can also be applied to subjective image testing. However, since the content captured is frequently changing and lacks fixed requirements, objective ROI selection methods are not suitable. Currently, most subjective ROI automatic selection methods involve locking and cropping regions with desired features, dividing the image into regions. However, due to the complexity and richness of captured images, different regions exhibit varying performance metrics. In subjective testing, issues such as incorrect ROI selection or failure to select ROIs may arise. If the phone image has image clarity issues, misclassification may occur, potentially misclassifying dense areas as flat areas, leading to incorrect noise or partial clarity assessments. Furthermore, severe color casts on the phone can result in incorrect ROI selection, affecting color module adjustments. Secondly, the size and position of the ROI are critical. A threshold set too low may result in adjacent or overlapping ROIs, while a threshold set too high may prevent adjacent objects from being selected as ROIs, even though those objects might be key areas of interest. Furthermore, it may also have an impact on the subsequent region of interest alignment stage, which places higher demands on image capture and will filter out many images when it comes to region of interest alignment.
[0043] This application discloses a method, apparatus, electronic device, and storage medium for determining the region of interest (ROI) of an image, which can improve the accuracy of determining the ROI of an image. These will be described in detail below.
[0044] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for determining the region of interest (ROI) of an image, as disclosed in an embodiment of this application. Figure 1The method for determining the region of interest in an image described herein is applicable to electronic devices such as smartphones, tablets, and laptops, and is not limited to any specific embodiment in this application.
[0045] like Figure 1 As shown, the method for determining the region of interest in this image may include the following steps:
[0046] 101. Obtain the first test image captured by the test equipment.
[0047] The electronic device acquires the first test image captured by the test equipment.
[0048] The testing equipment can be a camera device in the parameter testing phase. To acquire high-quality images, the camera device needs to adjust various parameters, such as white balance, color balance, and brightness. Therefore, the first test image captured by the testing equipment can be used to determine the region of interest (ROI) and to test the various parameters of the testing equipment based on the ROI.
[0049] The electronic device can communicate with the test equipment, and the test equipment can send the first test image to the electronic device via wireless communication methods such as Bluetooth and Wi-Fi.
[0050] 102. Input the first test image into the trained segmentation model for classification to obtain the second test image.
[0051] The electronic device inputs the first test image into a pre-trained segmentation model for classification to obtain a second test image, wherein the second test image includes multiple labeled regions corresponding to different object categories.
[0052] It should be noted that the electronic device can load the segmentation model locally, or the electronic device can download the segmentation model from the server; there are no specific limitations.
[0053] The segmentation model can be a convolutional neural network-based model for object detection, semantic segmentation, and instance segmentation of images. The segmentation model can predict the location and category of each object in a first test image, detect and extract the edges of objects belonging to different categories, thereby obtaining the labeled regions corresponding to each object, and outputting a second test image including multiple labeled regions. For example, the segmentation model can predict the location and category of each object in the first prediction image based on a fully convolutional network (FCN), and then detect and extract the edges of objects belonging to different categories based on a masked region convolutional network (Mask R-CNN). That is, the segmentation model can classify the objects in the first test image, and the second test image output by this segmentation model can include multiple regions corresponding to different object categories, i.e., regions containing multiple different categories of objects.
[0054] In the second test image, the multiple labeled regions corresponding to different object categories can be obtained by labeling the regions containing multiple different object categories separately. Specifically, the methods for labeling the regions containing multiple different object categories can be color labeling, number labeling, letter labeling, etc., without any specific limitation.
[0055] As an optional implementation, the second test image is obtained by the segmentation model color-coding the regions corresponding to each object in the first test image based on the category to which each object belongs in the first test image and a preset mapping relationship between category and color.
[0056] For example, the preset mapping relationship between categories and colors can be: when the object category is daytime sky, the color is light purple; when the object category is nighttime sky, the color is dark purple; when the object category is water surface, the color is blue; when the object category is lawn, the color is green, etc., but not limited to these.
[0057] The segmentation model can be used to predict the category of each object in the first test image, and to color-label the regions corresponding to each object in the first test image based on the category of each object in the first test image and the preset mapping relationship between category and color.
[0058] For example, a segmentation model identifies the location of a lawn in a first test image by detecting and extracting its edges. The region within the lawn's edge is then identified as the lawn's location. Based on a predefined mapping between categories and colors, the model determines that the lawn's color is green, and thus labels the corresponding area as green, resulting in the labeled area for the lawn. Therefore, the segmentation model can label regions corresponding to different categories of objects with different colors.
[0059] 103. Based on the target object category corresponding to the test index of the first test image, determine the target annotation region that matches the target object category from the second test image, and use it as the region of interest of the first test image.
[0060] The electronic device determines the target labeled region that matches the target object category from the second test image based on the target object category corresponding to the test index of the first test image, and uses it as the region of interest of the first test image.
[0061] The test metrics can be indicators used to test image quality, such as image sharpness, color, white balance, brightness, noise, detail, and sharpness. Since different objects have different image characteristics, it is necessary to identify the object that best reflects the test metric and designate the area containing that object as the region of interest.
[0062] As an optional implementation, before determining the target annotation region matching the target object category in the second test image based on the target object category corresponding to the test index of the first test image, and using it as the region of interest of the first test image, the electronic device may further perform the following steps:
[0063] Based on the mapping relationship between test indicators and object categories, the target object category corresponding to the test indicator of the first test image is determined; the mapping relationship between test indicators and object categories is determined based on the degree to which the image features of the object reflect the test indicators.
[0064] The degree to which an object's image features reflect the test metric can be based on user experience. For example, if the test metric is noise, the corresponding target object category could be objects with smooth, flat surfaces, such as walls, because smooth, flat surfaces are more likely to reflect unnecessary or redundant interference information. If the test metric is detail or sharpness, the corresponding target object category could be objects with dense image features, such as lawns or leaves, because dense objects are more likely to reflect image detail or sharpness. If the test metric is white balance, the target object category could be gray objects, such as white clothes or gray walls, as gray objects are more likely to reflect white balance parameters. If the test metric is highlights, the target object category could be bright objects, such as the daytime sky, as bright objects are more likely to reflect the degree of highlight.
[0065] The electronic device determines a target labeled region in the second test image that matches the target object category based on the test index of the first test image, and uses this region as the region of interest (ROI) of the first test image. For example, if the test index is detail and the target object category is grass, the target labeled region in the second test image that matches the grass can be the complete area corresponding to the grass, or it can be a randomly cropped portion of the complete area corresponding to the grass, and this target labeled region is used as the ROI of the first test image.
[0066] Optionally, the number of target labeled regions that the electronic device determines from the second test image that match the target object category can be one or more.
[0067] For example, if there is only one target labeled region, the electronic device may determine the region of interest (ROI) of the first test image from this target labeled region by randomly selecting a ROI of a specified size and / or shape within the target labeled region, or by determining a scaling factor and scaling down the target labeled region based on the scaling factor to obtain the ROI of the first test image. In summary, the ROI can be less than or equal to the target labeled region; for example, if the size of the target labeled region is A, the size of the ROI can be 0.5 times A.
[0068] For example, if there are multiple target labeled regions, the electronic device may determine the region of interest of the first test image from the multiple target labeled regions by randomly selecting one of the target labeled regions as the region of interest of the first test image; or, further, by selecting the region of interest of the first test image from the selected target labeled region.
[0069] By performing the above steps, the region of interest can be selected based on the target labeled area according to the required proportion, which can avoid problems such as the region of interest being too large or repetitive.
[0070] As another optional implementation, based on the target object category corresponding to the test index of the first test image, a target annotation region matching the target object category is determined from the second test image, which is then used as the region of interest of the first test image. The method further includes:
[0071] Calculate the coordinate data of the region of interest (ROI) in the first test image; based on the coordinate data, crop along the edge of the ROI of the first test image to obtain the ROI of the first test image.
[0072] By performing the above steps and cropping the region of interest based on the coordinate data of the region of interest in the first test image, the accuracy of obtaining the region of interest in the first test image can be improved.
[0073] In this embodiment, a first test image captured by a testing device is input into a pre-trained segmentation model for classification to obtain a second test image. The second test image includes multiple labeled regions corresponding to different object categories. Based on the target object category corresponding to the test index of the first test image, a target labeled region matching the target object category is determined from the multiple labeled regions as the region of interest (ROI) of the first test image. This embodiment first segments the first test image into labeled regions corresponding to different object categories using a segmentation model. Based on the target object category used to reflect the test index of the first test image, the ROI of the first test image is accurately located, improving the accuracy of determining the ROI of the image.
[0074] Please see Figure 2 , Figure 2 This is a flowchart illustrating another method for determining the region of interest in an image, as disclosed in an embodiment of this application. Figure 2 The described method for determining the region of interest in an image is applicable to electronic devices such as smartphones, tablets, and laptops, and is not limited to this application. Figure 2 As shown, the method includes the following steps:
[0075] 201. Obtain the first test image captured by the test equipment.
[0076] 202. Input the first test image into the trained segmentation model for classification to obtain the second test image.
[0077] The second test image includes multiple labeled regions, each corresponding to a different object category.
[0078] 203. Based on the target object category corresponding to the test index of the first test image, determine the target annotation region that matches the target object category from the second test image, and use it as the region of interest of the first test image.
[0079] The specific implementation methods for steps 201 to 203 can be found in the above embodiments.
[0080] 204. Obtain the first standard image captured by the comparison device.
[0081] The electronic device acquires a first standard image captured by the comparison device.
[0082] The comparison device can be a camera with its parameters fully calibrated, such as white balance, color balance, and brightness. This means the comparison device can acquire high-quality images that meet the user's needs. Therefore, the first standard image captured by the comparison device can serve as a benchmark. The electronic device can automatically adjust the first test image based on the first standard image; alternatively, the user can manually adjust the first test image based on the first standard image.
[0083] The electronic device can communicate with the comparison device, which can send the first standard image to the electronic device via wireless communication methods such as Bluetooth and Wi-Fi.
[0084] It should be noted that the first standard image acquired by the comparison device and the first test image acquired by the test device were obtained by shooting the same scene.
[0085] 205. Based on the first standard image, test the region of interest of the first test image.
[0086] The electronic device tests the region of interest of the first test image based on the first standard image.
[0087] As an optional implementation, the electronic device can align the first standard image and the first test image, determine the corresponding region of the region of interest in the first test image in the first standard image based on the coordinate position of the region of interest in the first test image in the first test image, and use it as the region of interest in the first standard image; and test the region of interest in the first test image based on the region of interest in the first standard image.
[0088] For example, the testing method for the region of interest (ROI) of the first test image is as follows: The electronic device can read test parameters such as saturation, color deviation, white balance, and signal-to-noise ratio of the ROI of the first standard image, and then read the test parameters such as saturation, color deviation, white balance, and signal-to-noise ratio of the ROI of the first test image. Based on the values of each test parameter of the first standard image, the values of each test parameter of the first test image are adjusted so that the values of each test parameter of the first test image are close to the values of the corresponding test parameters of the first standard image. For example, if the saturation of the ROI of the first standard image is A, and the saturation of the ROI of the first test image is B, and B is less than A, the saturation of the ROI of the first test image can be automatically increased to make it closer to A.
[0089] As another optional implementation, testing the region of interest of the first test image based on the first standard image may include the following steps:
[0090] The first standard image is input into the segmentation model to obtain the second standard image, which includes multiple colored labeled regions. The region of interest (ROI) of the first standard image is determined from the multiple labeled regions of the second standard image. Based on the ROI of the first standard image, the ROI of the first test image is tested.
[0091] By performing the above steps, both the first test image captured by the testing device and the first standard image captured by the comparison device are segmented using a segmentation model, and regions of interest are selected. The selected regions of interest are all for the same type of object, which improves the alignment rate of the regions of interest corresponding to the first test image and the first standard image, thereby improving the accuracy of testing the regions of interest of the first test image.
[0092] The segmentation model can be used to predict the category of each object in the first standard image, and to color-label the regions corresponding to each object in the first standard image based on the categories of each object in the first test image and a preset category-color mapping relationship. Therefore, the second standard image includes multiple colored labeled regions, with different colored standard regions corresponding to regions corresponding to different categories of objects.
[0093] One method for determining the region of interest (ROI) of the first standard image from multiple labeled regions of the second standard image is to determine the ROI of the first standard image from multiple labeled regions of the second standard image based on the test indicators of the first test image. Therefore, the electronic device can test the ROI of the first test image based on the ROI of the first standard image.
[0094] For example, if the testing metric for the first test image is detail, the electronic device can determine the region of interest (ROI) of the first standard image from the regions corresponding to objects such as grass and leaves, based on multiple labeled regions of the second standard image. Similarly, the electronic device can determine the ROI of the first test image from the regions corresponding to objects such as grass and leaves, based on multiple labeled regions of the second test image. Therefore, the ROI of the first test image and the ROI of the first standard image are determined based on the same testing metric, which can improve the effectiveness of image testing.
[0095] Optionally, the annotation color corresponding to the region of interest in the first standard image is the same as the annotation color corresponding to the region of interest in the first test image.
[0096] The consistent color coding indicates that the object category corresponding to the region of interest (ROI) in the first standard image is the same as the object category corresponding to the ROI in the first test image. Therefore, the ROIs in the first test image and the ROIs in the first standard image can be regions corresponding to the same type of object, which can further improve the test results.
[0097] Please see Figure 3 , Figure 3This is a user interface diagram of a method for determining the region of interest (ROI) of an image, as disclosed in an embodiment of this application. The test image selection area 301 is used to input a first test image captured by the test equipment, the comparison image selection area 302 is used to input a first standard image captured by the comparison equipment, and the segmentation model segmentation 303 is used to input the first test image and the first standard image into the segmentation model to obtain a second test image and a second standard image. The first column of test indicators 304 may include noise 3041, detail 3042, and sharpness 3043; the second column of test indicators 305 may include white balance 3051 and color 3052; and the third column of test indicators 306 may include brightness 3061. The user can determine the test indicator by clicking on any of the test indicators: noise 3041, detail 3042, sharpness 3043, white balance 3051, color 3052, and brightness 3061. The corresponding test result display area 307 after ROI selection can be used to output the ROI of the first test image, the ROI of the first standard image, and the test results of testing the ROI of the first test image based on the ROI of the first standard image.
[0098] In this embodiment, the first test image is first segmented into labeled regions corresponding to different object categories using a segmentation model. Based on the target object category used to reflect the test index of the first test image, the region of interest of the first test image is accurately located, which improves the accuracy of determining the region of interest of the image. Moreover, based on the first standard image captured by the comparison device, the region of interest of the first test image captured by the test device is tested, which improves the accuracy of image testing.
[0099] Please see Figure 4 , Figure 4 This is a flowchart illustrating another method for determining the region of interest in an image, as disclosed in an embodiment of this application. Figure 4 The described method for determining the region of interest in an image is applicable to electronic devices such as smartphones, tablets, and laptops, and is not limited to this application. Figure 4 As shown, the method includes the following steps:
[0100] 401. Obtain the first test image captured by the test equipment.
[0101] The implementation method of step 401 can refer to the above embodiments, and will not be described in detail here.
[0102] 402. Obtain multiple sample images.
[0103] The sample images include multiple images acquired based on different shooting parameters.
[0104] The images obtained based on different shooting parameters may include images with different resolutions, angles, brightness, and color casts. These images obtained based on different shooting parameters may be taken from the same scene.
[0105] Training on large datasets for segmentation of multiple images acquired with different shooting parameters can expand the types and forms that can be classified. This allows for accurate segmentation of object categories in images under different modes during subsequent testing, and enables the selection of regions of interest for testing in different modes.
[0106] 403. Based on multiple sample images, obtain multiple labeled images.
[0107] Multiple labeled images are obtained by labeling multiple sample images based on the user's classification requirements; each labeled image includes multiple user-labeled labeled regions.
[0108] The user's classification requirements can be determined based on the testing requirements of the image. Classification requirements can include requirements for object categories. For example, if the user only wants to test plants in the sample image, the electronic device can identify only the plants in the sample image and their positions in the sample image based on the user's classification requirements. If the user wants to test both plants and animals in the sample image, the electronic device can identify both plants and animals in the sample image and their positions in the sample image based on the user's classification requirements.
[0109] Users can manually label the regions corresponding to each object in multiple sample images. For example, the labeling method can be color labeling. Users can label the region corresponding to grass in the sample image as green and the region corresponding to water in the sample image as blue.
[0110] Optionally, the sample images can be images obtained by taking pictures of people; in the multiple labeled images obtained based on multiple sample images, the labels of the regions corresponding to the facial skin are different from the labels of the regions corresponding to the body skin. The segmentation model trained in this way can distinguish between facial skin and body skin very well, thereby avoiding errors when selecting the region of interest of the skin during summer shooting.
[0111] Optionally, the sample images can be images obtained by taking pictures of the cards used for objective testing. The segmentation model is trained based on the sample images and can be used to perform objective testing on the images.
[0112] 404. A segmentation model is trained based on multiple labeled images.
[0113] The electronic device trains a segmentation model based on a large amount of data from multiple labeled images. The training method for the segmentation model can be linear regression, gradient descent, softmax regression, etc., and the specific method is not limited.
[0114] It should be noted that steps 402 to 404 are methods for generating segmentation models. The electronic device in this embodiment can be used to generate segmentation models, or the electronic device can receive segmentation models generated on other electronic devices.
[0115] 405. Input the first test image into the trained segmentation model for classification to obtain the second test image.
[0116] The second test image includes multiple labeled areas.
[0117] 406. Based on the target object category corresponding to the test index of the first test image, determine the target annotation region that matches the target object category from the second test image, and use it as the region of interest of the first test image.
[0118] The implementation methods for steps 405 and 406 can refer to the above embodiments, and will not be described in detail here.
[0119] In this embodiment, the first test image is first segmented into labeled regions corresponding to different object categories using a segmentation model. Based on the target object category used to reflect the test index of the first test image, the region of interest of the first test image is accurately located, which improves the accuracy of determining the region of interest of the image. Moreover, by acquiring multiple sample images taken with different shooting parameters, a comprehensive sample set for training the segmentation model is collected, which improves the accuracy of training the segmentation model and the generalization ability of the segmentation model.
[0120] Please see Figure 5 , Figure 5 This is a schematic diagram of an image region of interest determination device disclosed in an embodiment of this application. This device can be applied to electronic devices such as smartphones, tablets, and laptops; however, this application does not limit its application to such devices. Figure 5 As shown, the region of interest determination device 500 for an image may include: an acquisition module 510, an input module 520, and a determination module 530;
[0121] The acquisition module 510 is used to acquire the first test image captured by the test equipment;
[0122] The input module 520 is used to input the first test image into the trained segmentation model for classification to obtain the second test image; the second test image includes multiple labeled regions corresponding to different object categories;
[0123] The determination module 530 is used to determine the target annotation region that matches the target object category in the second test image based on the target object category corresponding to the test index of the first test image, and use it as the region of interest of the first test image.
[0124] In one embodiment, the determining module 530 is further configured to determine the target object category corresponding to the test index of the first test image based on the mapping relationship between the test index and the object category, before determining the target annotation region matching the target object category from the second test image according to the target object category corresponding to the test index of the first test image, as the region of interest of the first test image; the mapping relationship between the test index and the object category is determined based on the degree to which the image features of the object reflect the test index.
[0125] In one embodiment, the second test image is obtained by the segmentation model color-coding the regions corresponding to each object in the first test image based on the category to which each object belongs in the first test image and a preset mapping relationship between category and color.
[0126] In one embodiment, the region of interest determination device 500 for the image further includes a test module;
[0127] The testing module is used to acquire a first standard image captured by a comparison device; and to test the region of interest in the first test image based on the first standard image.
[0128] In one embodiment, the testing module is further configured to input a first standard image into a segmentation model to obtain a second standard image, the second standard image including multiple colored labeled regions; determine the region of interest of the first standard image from the multiple labeled regions of the second standard image; and test the region of interest of the first test image based on the region of interest of the first standard image.
[0129] In one embodiment, the annotation color corresponding to the region of interest in the first standard image is the same as the annotation color corresponding to the region of interest in the first test image.
[0130] In one embodiment, the region of interest determination device 500 for the image further includes a training module;
[0131] The training module is used to acquire multiple sample images; the multiple sample images include multiple images acquired based on different shooting parameters; based on the multiple sample images, multiple labeled images are acquired; the multiple labeled images are obtained by labeling the multiple sample images based on the user's classification requirements; each labeled image includes multiple labeled regions labeled by the user; based on the multiple labeled images, a segmentation model is trained.
[0132] In this embodiment, a first test image captured by a testing device is input into a pre-trained segmentation model for classification to obtain a second test image. The second test image includes multiple labeled regions corresponding to different object categories. Based on the target object category corresponding to the test index of the first test image, a target labeled region matching the target object category is determined from the multiple labeled regions as the region of interest (ROI) of the first test image. This embodiment first segments the first test image into labeled regions corresponding to different object categories using a segmentation model. Based on the target object category reflecting the test index of the first test image, the ROI of the first test image is accurately located, improving the accuracy of determining the ROI of the image.
[0133] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.
[0134] like Figure 6 As shown, the electronic device 600 may include:
[0135] Memory 610 storing executable program code;
[0136] Processor 620 coupled to memory 610;
[0137] The processor 620 calls the executable program code stored in the memory 610 to execute any of the image region of interest determination methods disclosed in the embodiments of this application.
[0138] This application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by the processor, the processor implements any of the image region of interest determination methods disclosed in this application.
[0139] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0140] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0141] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0143] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.
[0144] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.
[0145] The foregoing has provided a detailed description of a method, apparatus, electronic device, and storage medium for determining the region of interest (ROI) of an image, as disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there may be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for determining the region of interest in an image, characterized in that, The method includes: Acquire the first test image captured by the test equipment; The first test image is input into a trained segmentation model for classification to obtain a second test image; the second test image includes multiple labeled regions corresponding to different object categories. Based on the mapping relationship between test indicators and object categories, the target object category corresponding to the test indicator of the first test image is determined; the mapping relationship between test indicators and object categories is determined based on the degree to which the image features of the object reflect the test indicators. Based on the target object category corresponding to the test index of the first test image, a target annotation region matching the target object category is determined from the second test image and used as the region of interest of the first test image. The target annotation region includes one or more regions. Based on the target annotation region, a portion of the region is cropped according to a preset ratio or randomly and used as the region of interest of the first test image.
2. The method according to claim 1, characterized in that, The second test image is obtained by the segmentation model after color-coding the regions corresponding to each object in the first test image based on the category to which each object belongs in the first test image and a preset mapping relationship between category and color.
3. The method according to claim 1, characterized in that, After determining a target labeled region in the second test image that matches the target object category based on the test index of the first test image, as the region of interest of the first test image, the method further includes: Acquire a first standard image captured by the comparison device; Based on the first standard image, the region of interest of the first test image is tested.
4. The method according to claim 3, characterized in that, The step of testing the region of interest in the first test image based on the first standard image includes: The first standard image is input into the segmentation model to obtain a second standard image, which includes multiple colored labeled regions. The region of interest of the first standard image is determined from multiple labeled regions of the second standard image; Based on the region of interest of the first standard image, the region of interest of the first test image is tested.
5. The method according to claim 4, characterized in that, The annotation color corresponding to the region of interest in the first standard image is the same as the annotation color corresponding to the region of interest in the first test image.
6. The method according to any one of claims 1-5, characterized in that, The method for generating the segmentation model includes: Acquire multiple sample images; the multiple sample images include multiple images acquired based on different shooting parameters; Based on the multiple sample images, multiple labeled images are obtained; the multiple labeled images are obtained by labeling the multiple sample images based on the user's classification requirements; each labeled image includes multiple labeled regions annotated by the user; The segmentation model is trained based on the multiple labeled images.
7. A device for determining the region of interest in an image, characterized in that, The device includes: The acquisition module is used to acquire the first test image captured by the test equipment; The input module is used to input the first test image into a pre-trained segmentation model for classification to obtain a second test image; the second test image includes multiple labeled regions corresponding to different object categories. The determination module is used to determine the target object category corresponding to the test index of the first test image based on the mapping relationship between the test index and the object category; the mapping relationship between the test index and the object category is determined based on the degree to which the image features of the object reflect the test index. Based on the target object category corresponding to the test index of the first test image, a target annotation region matching the target object category is determined from the second test image and used as the region of interest of the first test image. The target annotation region includes one or more regions. Based on the target annotation region, a portion of the region is cropped according to a preset ratio or randomly and used as the region of interest of the first test image.
8. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.