Image region determination method and device, electronic equipment and readable storage medium

Abstract

This application discloses an image region determination method, apparatus, electronic device, and readable storage medium, relating to the field of image processing technology. The method includes: acquiring an image to be recognized and a bounding box region determined by a target user on the image to be recognized; acquiring a first set of pixels corresponding to crowd objects in the image to be recognized; acquiring a second set of pixels corresponding to human shadows in the image to be recognized; obtaining a third set of pixels based on the first and second set of pixels; adjusting the bounding box range of the bounding box region based on the third set of pixels; and determining a target image region in the image to be recognized that matches the adjusted bounding box region. This can improve the accuracy of recognizing crowd objects and human shadows within the bounding box region.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image region determination method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Currently, with the development of electronic information technology, users can select a portion of an image and adjust that selected area. However, current methods for determining image regions have relatively low accuracy. Summary of the Invention

[0003] This application proposes an image region determination method, apparatus, electronic device, and readable storage medium.

[0004] In a first aspect, embodiments of this application provide an image region determination method, comprising: acquiring an image to be identified and a bounding region determined by a target user through a bounding operation on the image to be identified; acquiring a first set of pixels corresponding to crowd objects in the image to be identified; acquiring a second set of pixels corresponding to human shadows in the image to be identified; obtaining a third set of pixels based on the first set of pixels and the second set of pixels; adjusting the bounding range of the bounding region based on the third set of pixels, and determining a target image region in the image to be identified that matches the adjusted bounding region.

[0005] Secondly, embodiments of this application also provide an image region determination device, including: a first acquisition unit, a second acquisition unit, a third acquisition unit, a generation unit, and a target image region determination unit. The first acquisition unit is used to acquire an image to be recognized and a bounding region determined by a target user through a bounding operation on the image to be recognized; the second acquisition unit is used to acquire a first set of pixels corresponding to crowd objects in the image to be recognized; the third acquisition unit is used to acquire a second set of pixels corresponding to human shadows in the image to be recognized; the generation unit is used to obtain a third set of pixels based on the first set of pixels and the second set of pixels; the target image region determination unit is used to adjust the bounding range of the bounding region based on the third set of pixels, and determine a target image region in the image to be recognized that matches the adjusted bounding region.

[0006] Thirdly, embodiments of this application also provide an electronic device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the method described in the first aspect.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program code that can be invoked by a processor to execute the method described in the first aspect above.

[0008] The image region determination method, apparatus, electronic device, and readable storage medium provided in this application first acquire an image to be identified and a bounding area determined by a target user through a bounding operation on the image to be identified; then, a first set of pixels corresponding to crowd objects in the image to be identified is acquired; a second set of pixels corresponding to human shadows in the image to be identified is acquired; and a third set of pixels is obtained based on the first set of pixels and the second set of pixels; thereby, the bounding range of the bounding area is adjusted based on the third set of pixels to determine a target image region in the image to be identified that matches the adjusted bounding area. In this application, a third set of pixels is generated by acquiring a first set of pixels representing crowd objects and a second set of pixels representing human shadows. That is, the third set of pixels includes a set of pixels representing crowd objects and human shadows. By adjusting the bounding area based on the third set of pixels and obtaining the target image region, the accuracy of identifying crowd objects and human shadows in the bounding area can be improved.

[0009] Other features and advantages of the embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the embodiments of this application. The objects and other advantages of the embodiments of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 The following diagram illustrates an application scenario of the image region determination method provided in this application embodiment;

[0012] Figure 2 A flowchart of the image region determination method provided in an embodiment of this application is shown;

[0013] Figure 3 A flowchart of an image region determination method according to another embodiment of this application is shown;

[0014] Figure 4This illustration shows a schematic diagram of determining the set of fourth pixel points according to an embodiment of this application;

[0015] Figure 5 A flowchart of an image region determination method according to another embodiment of this application is shown;

[0016] Figure 6 A flowchart of an image region determination method according to another embodiment of this application is shown;

[0017] Figure 7 This diagram illustrates the effect of eliminating a target image region determined by the image region determination method provided in this application embodiment.

[0018] Figure 8 This illustration shows another effect of eliminating a target image region determined by the image region determination method provided in the embodiments of this application;

[0019] Figure 9 This illustration shows another effect of eliminating a target image region determined by the image region determination method provided in the embodiments of this application.

[0020] Figure 10 A structural block diagram of the image region determination device provided in an embodiment of this application is shown;

[0021] Figure 11 A structural block diagram of the electronic device provided in an embodiment of this application is shown;

[0022] Figure 12 A structural block diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. The components of the embodiments of the present application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without inventive effort are within the scope of protection of the present application.

[0024] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] Currently, with the development of electronic information technology, users can select specific regions of an image and adjust those regions. However, current methods for determining image regions have relatively low accuracy. Improving the accuracy of image region determination is a problem that urgently needs to be solved.

[0026] Currently, users can input the area of ​​an image that needs adjustment, for example, by smearing, circling, or dotting. The model can then determine the objects matching that area in the image, thus identifying the region requiring adjustment. For example, this model could be a panoramic segmentation model or an interactive segmentation model. A panoramic segmentation model can be used to segment an image, dividing pixels by object to obtain pixels corresponding to different targets, thus creating a segmentation map. An interactive segmentation model can use semantic and texture information to segment pixels representing objects within the user-input image region.

[0027] However, the inventors discovered in their research that methods that rely solely on panoramic recognition or interactive segmentation models to determine objects matching regions in an image, and thus identify the image regions requiring adjustment, perform poorly in recognizing human shadows within images. Furthermore, combining models to determine objects matching regions may fail to identify objects, resulting in situations where the model cannot match the desired image region even if the user inputs one. Therefore, the accuracy of image region identification is low.

[0028] Therefore, in order to solve or partially solve the above problems, this application provides an image region determination method, apparatus, electronic device, and readable storage medium.

[0029] Please see Figure 1 , Figure 1 The illustration shows an application scenario diagram of the image region determination method provided in the embodiments of this application, namely image region determination 100. The video generation scenario 100 may include an electronic device 110 and a cloud server 120, wherein the electronic device 110 is connected to the cloud server 120.

[0030] Electronic device 110 can connect to cloud server 120, which is also connected to the internet, by accessing the internet. Electronic device 110 can access the internet wirelessly, such as through wireless communication technologies like Wi-Fi or Bluetooth; it can also access the internet wiredly, such as through an RJ45 network cable or fiber optic cable.

[0031] Users can control the electronic device 110 to execute the image region determination method. For example, users can directly operate the electronic device 110 to control it to execute the image region determination method. Additionally, during the execution of the image region determination method, the electronic device 110 can also call the cloud server 120 to perform some steps, thereby reducing the computational burden on the electronic device 110. Detailed descriptions can be found in subsequent implementation methods.

[0032] Please see Figure 2 , Figure 2 A flowchart of an image region determination method according to an embodiment of this application is shown. This image region determination method can be applied to... Figure 1 The illustrated image region determination scenario uses an electronic device, specifically, the processor of the electronic device, as the execution entity for performing the image region determination method. This image region determination method may include steps S110 to S150.

[0033] Step S110: Obtain the image to be recognized and the bounding box area determined by the target user through a bounding box operation on the image to be recognized.

[0034] The image to be identified is the image whose target image region needs to be determined, which is the target image region subsequently determined from the image to be identified.

[0035] In some implementations, the electronic device may store images, allowing a target user to select one image from the stored images as the image to be identified. For example, the target user can open a specified application on the electronic device and select an image from that application as the image to be identified; the specified application could be a photo album.

[0036] Electronic devices can also be equipped with image acquisition components, so that the target user can acquire images through the image acquisition components as images to be recognized.

[0037] In addition, electronic devices can establish communication connections with other devices, so that target users can send images to be identified to the electronic device through other devices, enabling the electronic device to acquire the images to be identified.

[0038] After acquiring the image to be recognized, the electronic device can display it, for example, through a display component configured on the electronic device. The target user can then perform a selection operation on the image to be recognized to define a selected area. This selected area is the result of the target user's selection operation on the image to be recognized.

[0039] For example, the electronic device is configured with a touch-sensitive display screen, allowing the target user to directly interact with the touch-sensitive display screen to perform a selection operation on the image to be recognized. For instance, the target user can directly touch and swipe on the displayed image to be recognized, thereby determining the corresponding range based on the trajectory of the target user's touch and swipe, which can then be used as the selection area.

[0040] Optionally, the selection operation can also involve the target user inputting text information, which the electronic device then performs semantic recognition on to obtain a selection area for the image to be recognized. For example, the text information could be "the white teacup on the left table," which would determine the range of the white teacup in the image to be recognized as the selection area.

[0041] Optionally, the electronic device can also be equipped with a microphone, so that the selection operation can also be performed by the target user speaking, thereby the electronic device acquiring the target user's voice, converting the voice into text information, and then performing semantic recognition on the text information to obtain the selected area.

[0042] Step S120: Obtain the first set of pixels corresponding to the crowd objects in the image to be identified.

[0043] In some implementations, the image to be identified may include a crowd of people, such as multiple individuals. The crowd may consist of one or more individuals, for example, multiple individuals located at different positions within the image. Therefore, the crowd can be extracted from the image. It is understood that the image to be identified is composed of multiple pixels, allowing for the identification of each pixel to extract the first set of pixels corresponding to the crowd.

[0044] In some implementations, a crowd segmentation algorithm can be used to extract the first set of pixels corresponding to the crowd objects from the image to be identified.

[0045] Optionally, a first masking image can be generated based on the first set of pixels corresponding to the crowd objects in the image to be identified. This first masking image is the crowd masking image. For example, in the first masking image, the color parameter of each pixel in the first set of pixels can be set to one value, while the pixels in the image to be identified other than the first pixel can be set to another value.

[0046] The first set of pixels corresponding to the crowd objects in the image to be identified can be obtained locally on the electronic device; or it can be obtained through a cloud server. For details, please refer to the following embodiments.

[0047] Optionally, the obtained first set of pixels may also correspond to label information, which can be used to characterize the object represented by the corresponding set of pixels. For example, the label information corresponding to the first set of pixels can indicate that the first set of pixels is used to characterize a crowd of people.

[0048] Step S130: Obtain the set of second pixel points corresponding to the human figure shadow in the image to be identified.

[0049] Similarly, the image to be identified may also include human shadows, for example, one human shadow; or multiple human shadows. Therefore, further, the human shadows can be extracted from the image to be identified. Specifically, each pixel can be identified to extract the second set of pixels corresponding to the human shadows from the image to be identified.

[0050] In some implementations, a second set of pixels corresponding to the shadow of a person can be extracted from the image to be identified using shadow detection and segmentation algorithms.

[0051] Optionally, a second mask image can be generated based on the second set of pixels corresponding to the human shadow in the image to be identified. This second mask image is the shadow mask image. For example, in the first mask image, the color parameter of each pixel in the second set of pixels can be set to one value, while the pixels in the image to be identified other than the first pixel can be set to another value.

[0052] Similarly, the set of second pixels corresponding to the human shadow in the image to be identified can be obtained locally on the electronic device; or the set of second pixels corresponding to the human shadow in the image to be identified can be obtained through a cloud server. For details, please refer to the following embodiments.

[0053] Optionally, the obtained second set of pixels may also correspond to label information, which can be used to characterize the object represented by the corresponding set of pixels. For example, the label information corresponding to the second set of pixels can indicate that the second set of pixels is used to characterize a portrait shadow.

[0054] Step S140: Obtain a third set of pixels based on the first set of pixels and the second set of pixels.

[0055] After obtaining the first set of pixels and the second set of pixels, a third set of pixels can be obtained based on the first set of pixels and the second set of pixels. For example, the first set of pixels and the second set of pixels can be merged to obtain the third set of pixels.

[0056] For example, the image to be identified can be segmented first to obtain a fourth set of pixels in the image to be identified to represent different objects. Then, the first set of pixels and the second set of pixels can be fused with the obtained fourth set of pixels to generate a third set of pixels. For a detailed description, please refer to the following embodiments.

[0057] In the embodiments provided in this application, the third pixel set generated by fusion includes a first pixel set for representing crowd objects and a second pixel set for representing human shadows, so that the target image region can be determined based on the third pixel set with higher accuracy.

[0058] Similarly, a third mask image can be generated based on a third set of pixels, which can be used to represent a segmentation mask image. When a third set of pixels is generated by fusing the first set of pixels, the second set of pixels, and the resulting fourth set of pixels, this third mask image can include a first set of pixels represented by a color parameter of one value, a second set of pixels represented by a color parameter of another value, and a set of pixels representing objects other than crowds and human shadows, represented by a color parameter of yet another value.

[0059] Step S150: Adjust the selection range of the selected area based on the third pixel set, and determine the target image area in the image to be identified that matches the adjusted selected area.

[0060] Therefore, after obtaining the third set of pixels, the selection range of the framed region can be adjusted based on the third set of pixels, thereby determining the target image region in the image to be recognized that matches the adjusted framed region.

[0061] It is understood that there may be at least one crowd object and at least one human shadow, thus each crowd object corresponds to a first set of pixels, and each human shadow corresponds to a second set of pixels. Therefore, in some embodiments, various cases can be determined based on whether the selected area includes the first or second set of pixels, and the ratio of the number of pixels in the selected area to the total number of pixels in the selected area. This allows for adjustments to the selected area under different circumstances, and then the target image region in the image to be identified that matches the adjusted selected area is determined. For detailed descriptions, please refer to subsequent embodiments.

[0062] It is understandable that the target image region in the image to be recognized that matches the adjusted bounding box region can be the region corresponding to the same pixels in the image to be recognized as the target image region.

[0063] The image region determination method provided in this application first obtains the image to be identified and the bounding area determined by the target user through a bounding operation on the image to be identified; then, it obtains a first set of pixels corresponding to crowd objects in the image to be identified; it obtains a second set of pixels corresponding to human shadows in the image to be identified; and then, it obtains a third set of pixels based on the first set of pixels and the second set of pixels; thereby, it adjusts the bounding range of the bounding area based on the third set of pixels to determine the target image region in the image to be identified that matches the adjusted bounding area. In this application, the first set of pixels representing crowd objects and the second set of pixels representing human shadows are obtained respectively, and then the third set of pixels is generated. That is, the third set of pixels includes the set of pixels representing crowd objects and human shadows. Therefore, by adjusting the bounding area based on the third set of pixels, the target image region can be obtained, which can improve the recognition accuracy of crowd objects and human shadows in the bounding area.

[0064] Please see Figure 3 , Figure 3 A flowchart of an image region determination method according to an embodiment of this application is shown. This image region determination method can be applied to... Figure 1 The illustrated image region determination scenario uses an electronic device, specifically, the processor of the electronic device, as the execution entity for performing the image region determination method. This image region determination method may include steps S210 to S260.

[0065] Step S210: Obtain the image to be recognized and the bounding box area determined by the target user through a bounding box operation on the image to be recognized.

[0066] Step S220: Obtain the first set of pixels corresponding to the crowd objects in the image to be identified.

[0067] Step S230: Obtain the set of second pixel points corresponding to the human figure shadow in the image to be identified.

[0068] Step S240: Obtain a third set of pixels based on the first set of pixels and the second set of pixels.

[0069] Steps S210 to S240 have been described in detail in the foregoing embodiments and will not be repeated here.

[0070] Step S250: In the third pixel set, find the first pixel set and the second pixel set in which at least some pixels are within the selected area, and use them as the fourth pixel set.

[0071] As described above, the image to be identified may also include crowds and human shadows. Therefore, in the embodiments provided in this application, if there is at least one crowd and at least one human shadow, each crowd can correspond to a first set of pixels, and each human shadow can correspond to a second set of pixels. That is, the third set of pixels includes multiple first sets of pixels representing crowds and multiple second sets of pixels representing human shadows. It is understood that after obtaining the selected area, the selection range of the selected area can be adjusted based on the set of pixels in the third set that are within the selected area. Therefore, the first set of pixels and the second set of pixels in the third set that have at least some pixels within the selected area can be searched first and second, respectively, as the fourth set of pixels.

[0072] It should be noted that a set of first or second pixels may not be entirely within the selected area. However, for a set of first pixels, if only a portion of its pixels are within the selected area, then the set of first pixels can be identified as being within the selected area. Similarly, for a set of second pixels, if only a portion of its pixels are within the selected area, then the set of second pixels can be identified as being within the selected area.

[0073] For example, please refer to Figure 4 , Figure 4 A schematic diagram illustrating the determination of the fourth pixel set provided in an embodiment of this application is shown. Figure 4The image 400 to be identified shown includes a bounding area 410, a first set of pixels 420, a first set of pixels 430, and a second set of pixels 440. That is, in... Figure 4 The third set of pixels includes the first set of pixels 420, the first set of pixels 430, and the second set of pixels 440.

[0074] from Figure 4 As can be seen, each pixel in the first pixel set 420 is outside the selected area 410, while some pixels in the second pixel set 430 are within the selected area 410. Furthermore, the first pixel set 440 is entirely within the selected area 410. Therefore, it can be determined that there exists a first pixel set and a second pixel set, including the second pixel set 430 and the first pixel set 440, where at least some pixels are within the selected area. In other words, the second pixel set 430 and the first pixel set 440 can be considered as the fourth pixel set.

[0075] Step S260: Adjust the selection range of the selected area based on the fourth pixel set, and determine the target image area in the image to be identified that matches the adjusted selected area.

[0076] It is understandable that the set of pixels in the third pixel set, excluding the fourth pixel set, is completely outside the selected area. In other words, the target user is highly unlikely to want to determine the subsequent target image region based on the set of pixels in the third pixel set excluding the fourth pixel set. Therefore, the selection range of the selected area can be adjusted based on the fourth pixel set to determine the target image region in the image to be recognized that matches the adjusted selected area.

[0077] In some implementations, the selection range of the frame can be adjusted based on the proportion of each pixel in the fourth pixel set that is within the selected area. Specifically, step S260 may include steps S261 to S265.

[0078] Step S261: Obtain a first ratio corresponding to each of the fourth pixel point sets, wherein the first ratio is the ratio between the number of pixels in the fourth pixel point set that are within the selected area and the total number of pixels in the fourth pixel point set.

[0079] As explained above, there can be multiple sets of fourth pixel points, for example... Figure 4In the example shown, there are two sets of fourth pixels: a second set of pixels 430 and a first set of pixels 440. Therefore, a first ratio can be obtained for each set of fourth pixels, where the first ratio is the ratio between the number of pixels in the fourth pixel set that are within the selected area and the total number of pixels in the fourth pixel set.

[0080] For example, continue with Figure 4 Taking the second pixel set 430 and the first pixel set 440 as examples of the fourth pixel set, the following description is provided. Specifically, for the first pixel set 440 as the fourth pixel set, the number of pixels within the selected area is the same as the total number of pixels in the fourth pixel set; therefore, the first ratio corresponding to the first pixel set 440 as the fourth pixel set is 100%. Furthermore, for the second pixel set 430 as the fourth pixel set, the number of pixels within the selected area is, for example, x1, and the total number of pixels in the fourth pixel set is, for example, x2; therefore, the first ratio corresponding to the second pixel set 430 as the fourth pixel set is x1 / x2. For example, x1 / x2 can be 35%.

[0081] Step S262: If at least one of the first ratios is greater than or equal to the first threshold, obtain the sum of the number of pixels in each of the fourth pixel point sets to obtain the first value.

[0082] Step S263: Obtain the total number of pixels corresponding to the selected area, as the second value.

[0083] Step S264: Adjust the selection range of the selected area based on the second ratio of the first value to the second value.

[0084] Furthermore, the adjustment of the selection range of the selected area can be determined based on the magnitude of each obtained first ratio. Specifically, each first ratio can be compared with a first threshold. If at least one first ratio is greater than or equal to the first threshold, the sum of the number of pixels in each of the fourth pixel sets can be obtained to obtain a first value. For example, the second pixel set 430 and the first pixel set 440 are used as the fourth pixel set. For the second pixel set 430 as the fourth pixel set, the sum of the number of pixels in this fourth pixel set is x2; for the first pixel set 440 as the fourth pixel set, the sum of the number of pixels in this fourth pixel set is x3. Therefore, the first value can be represented as x2 + x3.

[0085] The first threshold can be preset, for example, the first threshold can be set to 85%.

[0086] Additionally, the total number of pixels corresponding to the selected area can be obtained as a second value. For example, xk can be used to represent the total number of pixels corresponding to the selected area, i.e., xk is the second value.

[0087] Therefore, by comparing the first value and the second value, the proportion of the object corresponding to the fourth pixel set in the selected area can be characterized, and the selection range of the selected area can be adjusted based on the second ratio of the first value and the second value. Specifically, step S264 may include step S2641.

[0088] Step S2641: When the second ratio is greater than or equal to the second threshold, adjust the selection range of the selected area to the area corresponding to the fourth pixel set.

[0089] In some implementations, the selection range of the bounding box region can be adjusted by comparing the second ratio with a second threshold. Specifically, when the second ratio is greater than or equal to the second threshold, it indicates that the proportion of the fourth pixel set in the bounding box region is larger, meaning that the target user is more likely to want to select the object corresponding to the fourth pixel set. Therefore, the selection range of the bounding box region can be directly adjusted to the region corresponding to the fourth pixel set. The second threshold can be preset, for example, it can be 1 / 3.

[0090] It is understandable that, due to algorithmic segmentation issues, pixels at the edges of the fourth pixel set may have pixels of other objects attached to them. Therefore, the pixels at the edges of the fourth pixel set can be further refined into smaller segments, and then the selection area of ​​the bounding box can be adjusted to the region corresponding to the refinedly segmented fourth pixel set. Specifically, step S2641 may also include steps S2642 and S2643.

[0091] Step S2642: If the second ratio is greater than or equal to the second threshold, the pixels at the edge are refined from the fourth pixel set using a saliency segmentation network.

[0092] Step S2643: Adjust the selection range of the selected area to the area corresponding to the fourth pixel set after fine processing.

[0093] In some implementations, when the second ratio is greater than or equal to the second threshold, the pixels at the edges of the fourth pixel set can be refined using a saliency segmentation network. For example, when the first pixel set is used as the fourth pixel set, the pixels at the edges of this fourth pixel set can be included as pixels corresponding to hair. It is understood that hair is generally difficult to segment; therefore, in the embodiments provided in this application, refining the hair pixels using a saliency segmentation network can improve the accuracy of the refined fourth pixel set.

[0094] Among them, fine-grained processing can be fine-grained segmentation processing.

[0095] For example, a saliency segmentation network can be a U-2-Net network.

[0096] Therefore, the selection area of ​​the bounding box can be adjusted to the region corresponding to the fourth pixel set after fine processing. Subsequent determination of the target image region in the image to be recognized that matches the adjusted bounding box can further improve the accuracy of the target image region.

[0097] It is understandable that in the image to be identified, there may be some objects that are attached to the person object, but the target user may not notice these objects when determining the selection area, and thus the pixels corresponding to these objects may be outside the user-defined selection area. However, in order to improve the accuracy of determining the target image area, when adjusting the selection range, the pixels corresponding to the objects that are attached to the person object in the fourth pixel set can also be considered. Specifically, step S2641 may also include steps S2644 to S2646.

[0098] Step S2644: Determine the set of fifth pixels corresponding to a specified part of the person object in the set of fourth pixels.

[0099] Step S2645: Based on the image to be identified, find the set of sixth pixels that have an attachment relationship with the set of fifth pixels.

[0100] Step S2646: Adjust the selection range of the selected area to the area corresponding to the fourth pixel set and the sixth pixel set.

[0101] In some implementations, the fifth pixel set corresponding to a specified part of the person object in the fourth pixel set can be determined first. For example, the specified part can be the palm, thereby determining the fifth pixel set corresponding to the palm of the person object in the fourth pixel set.

[0102] Furthermore, the image to be identified can be searched for objects that have an attachment relationship with the fifth pixel set. For example, in the image to be identified, the object with an attachment relationship with the fifth pixel set is a toy held in the palm of a person's hand. Therefore, the pixel set corresponding to the toy in the image to be identified can be used as the sixth pixel set. Then, the selection area of ​​the bounding box is adjusted to include the area corresponding to the fourth pixel set and the sixth pixel set. The adjusted bounding box can then be used as a target segmentation mask.

[0103] Therefore, even if the user ignores the objects held by the person and fails to select these objects into the selection area, the image region determination method provided in this application embodiment can still merge and supplement the objects held by the person that the user ignored into the selection area, thereby improving the accuracy of the selection area and thus improving the accuracy of subsequent determination of the target image region.

[0104] In addition, the second ratio may be less than the second threshold, so step S264 may also include step S2647.

[0105] Step S2647: If the second ratio is less than the second threshold, the selection range of the selected area is not adjusted.

[0106] In other words, when the second ratio is less than the second threshold, it indicates that the proportion of the fourth pixel set in the selected area is small, meaning the target user is more likely not to select the object corresponding to the fourth pixel set. Therefore, the selected area determined by the target user can be retained without adjustment. That is, the user-determined selected area can be used as the target segmentation mask at this time.

[0107] Optionally, if the second ratio is less than the second threshold, the label information corresponding to the selected area can be set to "unknown" to indicate that the object in the selected area is unknown.

[0108] In addition to the case where at least one of the first ratios is greater than or equal to the first threshold, there may be other different magnitude relationships between multiple first ratios and the first threshold. Therefore, the selection range of the selected area can be adjusted differently under these other different magnitude relationships. Specifically, step S260 may also include steps S266 and S267.

[0109] Step S266: When the first ratio satisfies the target condition, the target object matching the selected region is identified from the image to be identified through interactive segmentation. The target condition includes a first condition or a second condition. The first condition includes that each first ratio is less than the third threshold. The second condition includes that each first ratio is less than the first threshold and the fourth pixel set includes at least one first pixel set. The third threshold is less than the first threshold.

[0110] Step S267: Use the image region corresponding to the target object as the target image region.

[0111] In some implementations, the relationship between each first ratio and a first threshold can be used to determine whether the first ratio satisfies the target condition. It should be noted that if the first ratio satisfies the target condition, it can be considered that the ratio of the number of pixels in the fourth pixel set within the selected area to the total number of pixels in that fourth pixel set is low. Therefore, this indicates that the user is highly unlikely to want to select any object corresponding to any of the fourth pixel sets.

[0112] Therefore, if the first ratio meets the target condition, the selected region can be interactively segmented directly to identify the target object matching the selected region from the image to be identified. Then, the image region corresponding to the target object is taken as the target image region. That is, the selected region can be identified and analyzed through interactive segmentation to obtain the image region corresponding to the target object in the selected region.

[0113] Optionally, after identifying the target object matching the selected region from the image to be identified through interactive segmentation, the pixels at the edge of the pixel set of the target object can be refined, so that the image region corresponding to the refined target object is taken as the target image region.

[0114] Interactive segmentation can be accomplished through an interactive segmentation model, which can be used to segment the pixels of objects in the user-input bounding box region from the image to be recognized using semantic and texture information.

[0115] It should be noted that interactive segmentation can only obtain the image region corresponding to the target object. However, the label information of the target object represented by this target image region cannot be obtained through interactive segmentation. That is, interactive segmentation can identify the target object matching the selected region; however, the label information of the target object, such as crowds, human shadows, buildings, vehicles, or plants, cannot be obtained through interactive segmentation.

[0116] The target condition may include a first condition or a second condition. That is, the first ratio only needs to satisfy either the first or second condition to be considered to meet the target condition. In the embodiments provided in this application, the first condition includes each of the first ratios being less than the third threshold, wherein the third threshold can be a preset value, and the third threshold is less than the first threshold. For example, the first threshold is 85%, and the third threshold is 5%.

[0117] Furthermore, the second condition includes that each of the first ratios is less than the first threshold and the fourth pixel set includes at least one first pixel set. In other words, the second condition includes that each of the first ratios is less than the first threshold and the fourth pixel set includes at least one first pixel set used to represent a crowd object.

[0118] Additionally, it should be noted that if the selected area does not include any third set of pixels, that is, each first ratio is 0, and thus each first ratio is less than the third threshold, then it can be determined that the first ratio meets the target condition, and thus the process jumps to the interactive segmentation to identify the target object matching the selected area from the image to be identified.

[0119] In some implementations, the electronic device can locally deploy a model for performing interactive segmentation, such as an interactive segmentation model, so that the target object matching the selected region can be identified from the image to be identified locally by the interactive segmentation model. Alternatively, if the first ratio meets the target condition, the electronic device can upload the image to be identified and the selected region to a cloud server, so that the target object matching the selected region can be identified from the image to be identified by the interactive segmentation model deployed on the cloud server. After obtaining the target object, the target object is then sent to the electronic device, thereby reducing the storage and transportation pressure on the electronic device for deploying and running the interactive segmentation model locally, and improving processing speed. Furthermore, after obtaining the target object sent by the cloud server, the electronic device can also initiate a data destruction request to the cloud server, so that the cloud server can respond to the data destruction request by cleaning up and deleting the image to be identified and the data generated during the acquisition of the target object, ensuring the privacy of the target user and improving data security. Optionally, the electronic device can also initiate a data destruction request to the cloud server after obtaining the target image region; this embodiment of the application does not specifically limit this.

[0120] In addition, in some embodiments, besides the case where at least one of the first ratios is greater than or equal to the first threshold, and the case where the first ratio satisfies the target condition, there may also be a case where the first ratio is between the first threshold and the third threshold. Specifically, step S260 may also include step S268.

[0121] Step S268: When each of the first ratios is less than the first threshold and greater than the third threshold, and the fourth pixel set includes at least one first pixel set, generate a prompt message and return to execute the process of obtaining the bounding area determined by the target user for the image to be identified and subsequent steps, wherein the prompt message is used to prompt the user to redefine the expanded bounding area, and the third threshold is less than the first threshold.

[0122] When each of the first ratios is less than the first threshold and greater than the third threshold, and the fourth pixel set includes at least one first pixel set (i.e., when each of the first ratios is less than the first threshold and greater than the third threshold, and the fourth pixel set includes at least one first pixel set representing a crowd object), a prompt message can be generated and the process can return to obtaining the bounding box area determined by the target user for the image to be identified and subsequent steps. The prompt message is used to prompt the user to redefine the expanded bounding box area, and the third threshold is less than the first threshold.

[0123] For example, the prompt message can be displayed on the screen of an electronic device, such as in text form. The prompt message can also be played through the speaker of an electronic device, such as by playing the language corresponding to the text message through the speaker.

[0124] Step S265: Determine the target image region in the image to be identified that matches the adjusted box selection region.

[0125] Understandably, the adjusted bounding box determines the set of pixels included, which can also be called the target segmentation mask. Furthermore, the adjusted bounding box can also determine the edge contour, which can be the envelope formed by the edge pixels in each pixel set within the bounding box. Moreover, the label information corresponding to each pixel set in the bounding box can be obtained. Then, based on the target segmentation mask, edge contour, and label information, the target image region is determined from the image to be recognized, improving the accuracy of target image region determination.

[0126] It should be noted that the target segmentation mask, edge contour lines, and label information can be used to cooperate with subsequent algorithm models for further processing. That is, the image region determination method provided by the embodiments of this application can provide target segmentation mask, edge contour lines, and label information for subsequent further processing.

[0127] The image region determination method provided in this application improves the accuracy of identifying crowd objects in the image to be identified. It also achieves intelligent bounding box selection of human shadows in the image to be identified. Furthermore, through a saliency segmentation algorithm, it refines the edge pixels in the fourth pixel set, improving the accuracy of the refined fourth pixel set. It can also intelligently merge and supplement objects held by crowd objects that the user has overlooked into the bounding box area, improving the accuracy of the bounding box area and thus improving the accuracy of subsequent target image region determination. This enhances the intelligence of determining objects in the bounding box area, achieving more intelligent bounding box selection without requiring additional user operations, resulting in a more accurate target image region.

[0128] Please see Figure 5 , Figure 5 A flowchart of an image region determination method according to an embodiment of this application is shown. This image region determination method can be applied to... Figure 1 The illustrated image region determination scenario uses an electronic device, specifically, the processor of the electronic device, as the execution entity for performing the image region determination method. This image region determination method may include steps S310 to S370.

[0129] Step S310: Obtain the image to be recognized and the bounding box area determined by the target user through a bounding box operation on the image to be recognized.

[0130] Step S320: Obtain the first set of pixels corresponding to the crowd objects in the image to be identified.

[0131] Step S330: Obtain the set of second pixel points corresponding to the human figure shadow in the image to be identified.

[0132] Steps S310 to S330 have been described in detail in the foregoing embodiments and will not be repeated here.

[0133] Step S340: Perform object segmentation on the image to be identified to obtain a set of seventh pixel points in the image to be identified that represent different objects.

[0134] Step S350: Update the object corresponding to the seventh pixel in the seventh pixel set that matches the shape of the first pixel set to a crowd object, and update the object corresponding to the pixel in the seventh pixel set that matches the shape of the second pixel set to a portrait shadow.

[0135] Step S360: Use the updated set of seventh pixels as the set of third pixels.

[0136] Object segmentation can be performed on the image to be identified to obtain a set of seventh pixels representing different objects in the image. For example, panoramic segmentation can be performed on the image to be identified; for instance, panoramic segmentation can be achieved using a panoramic segmentation model. Through panoramic segmentation, a set of seventh pixels representing different objects in the image to be identified can be obtained. For example, the set of seventh pixels can include multiple different objects, and each object can correspond to a set of pixels.

[0137] Optionally, the label information of each object in the seventh pixel set can also be obtained, and the label information can be used to represent the object corresponding to each pixel set.

[0138] In some implementations, a segmented mask image of the image to be identified can also be obtained through panoramic segmentation. Specifically, different color parameters can be set for the pixel sets of different objects in the seventh pixel set to generate the segmented mask image.

[0139] Furthermore, label information can be represented using dictionaries. For example, a dictionary representing a person object can be defined as {"id": 2, "category_id": 0}, where "id": 2 indicates that the color parameter of the pixel set of the object with "category_id": 0 is set to 2; and "category_id": 0 represents the person object. If another pixel set of a person object exists in the segmentation mask image, then the dictionary for that other person object can be represented as {"id": 3, "category_id": 0}.

[0140] Then, the objects corresponding to the seventh pixels in the seventh pixel set that match the shape of the first pixel set can be updated to crowd objects, and the objects corresponding to the pixels in the seventh pixel set that match the shape of the second pixel set can be updated to portrait shadows. In other words, the segmented mask image can be fused and updated using the first pixel set and the second pixel set.

[0141] For example, a set of pixels whose shape matches the seventh pixel can be determined from the first set of pixels, thereby updating all objects corresponding to the set of pixels in the seventh set whose shape matches the first set of pixels to crowd objects. For instance, the color parameters of all pixels in the seventh set whose shape matches the first set of pixels can be set to 20, and then the corresponding label information {"id": 20, "category_id": 100} can be added. Here, "category_id": 100 can be used to represent the object as a crowd object.

[0142] Similarly, the set of pixels that matches the shape of the seventh pixel can be determined by the set of second pixels, and then the objects corresponding to the set of pixels in the seventh pixel set that match the shape of the second pixel set can be updated to portrait shadows.

[0143] The updated set of seventh pixels can then be used as the set of third pixels. It is understood that this set of third pixels can include pixel sets corresponding to multiple objects, such as crowds, human shadows, and various objects obtained through panorama segmentation.

[0144] It should be noted that the electronic device can locally deploy a model for performing panoramic segmentation, such as a panoramic segmentation model, and can also deploy crowd segmentation algorithms and shadow detection and segmentation algorithms. Therefore, steps S320 to S360 described above can be performed locally on the electronic device to obtain the third pixel set.

[0145] Alternatively, a model for performing panoptic segmentation, such as a panoptic segmentation model, can be deployed on the cloud server. Crowd segmentation algorithms and shadow detection and segmentation algorithms can also be deployed. Thus, the electronic device can upload the image to be recognized to the cloud server, where steps S320 to S360 are executed. After obtaining the third pixel set, the cloud server sends the third pixel set to the electronic device.

[0146] Optionally, embodiments of this application can further expand the types of objects that can be identified in the panoramic segmentation model, thereby improving the effect of obtaining segmented masked images through the panoramic segmentation model. Alternatively, segmentation based on open cue words can replace panoramic segmentation.

[0147] Optionally, crowd segmentation algorithms, as well as shadow detection and segmentation algorithms, can be integrated into the panoramic segmentation algorithm to improve the execution efficiency of the image region determination method.

[0148] Optionally, before uploading the image to be identified to the cloud server, the electronic device can also decode and scale the image, thereby improving the efficiency of subsequent processing.

[0149] It should be noted that the execution order of steps S340, S320, and S330 is not specifically limited; they can be executed sequentially or in parallel. It is only necessary to ensure that steps S340, S320, and S330 are executed before step S350.

[0150] Step S370: Adjust the selection range of the framed region based on the third pixel set, and determine the target image region in the image to be identified that matches the adjusted framed region.

[0151] Step S370 has been described in detail in the foregoing embodiments and will not be repeated here.

[0152] In the image region determination method provided in this application embodiment, the objects corresponding to the seventh pixels in the seventh pixel set whose shapes match the first pixel set are updated to crowd objects, and the objects corresponding to the pixels in the seventh pixel set whose shapes match the second pixel set are updated to portrait shadows; the updated seventh pixel set is used as the third pixel set. This can improve the accuracy of each object included in the obtained third pixel set and the pixel set corresponding to each object, thereby improving the accuracy of subsequently obtaining the target image region.

[0153] Please see Figure 6 , Figure 6 A flowchart of an image region determination method according to an embodiment of this application is shown. This image region determination method can be applied to... Figure 1 The illustrated image region determination scenario uses an electronic device, specifically, the processor of the electronic device, as the execution entity for performing the image region determination method. This image region determination method may include steps S410 to S480.

[0154] Step S410: Obtain the image to be recognized.

[0155] First, the electronic device can acquire the image to be recognized.

[0156] Step S420: Shadow detection, panoramic segmentation, and crowd segmentation.

[0157] Then, panoramic segmentation, shadow detection, and crowd segmentation can be performed on the image to be recognized. Specifically, panoramic segmentation yields a segmentation mask image, which is then fused and updated using the second set of pixels obtained from shadow detection and the first set of pixels obtained from crowd segmentation to obtain the fused and updated segmented image.

[0158] Specifically, panoramic segmentation of the image to be recognized can be achieved through a panoramic segmentation model; shadow detection and segmentation algorithms can be used to perform shadow detection on the image to be recognized; and crowd segmentation algorithms can be used to perform crowd segmentation on the image to be recognized.

[0159] Optionally, the shadow detection, panoramic segmentation, and crowd segmentation described above can be performed locally on the electronic device or on a cloud server. This application embodiment does not impose specific limitations.

[0160] Step S430: Obtain label information and segmentation mask image.

[0161] The above steps yield the fused and updated segmented mask image. Optionally, the label information corresponding to each object in the segmented mask image can also be obtained, for example, by representing the label information in the form of a dictionary. The set of pixels of each object in the segmented mask image can then be used as the third set of pixels.

[0162] Step S440: Obtain the selected area.

[0163] It can also obtain the selected area determined by the target user through a selection operation on the image to be recognized.

[0164] Step S450: Adjust the selection range of the selected area based on the first ratio.

[0165] Alternatively, a first set of pixels and a second set of pixels that have at least some pixels within the selected area can be found in the third set of pixels and used as the fourth set of pixels.

[0166] Therefore, the selection range of the selected area can be adjusted based on the first ratio. The first ratio is the ratio between the number of pixels in the fourth pixel set that are within the selected area and the total number of pixels in the fourth pixel set.

[0167] In some implementations, there can be multiple sets of fourth pixels, so that each set of fourth pixels can correspond to a first ratio. Each first ratio can be compared with a first threshold and a third threshold, wherein the first threshold is greater than the third threshold. For example, the first threshold can be 85% and the third threshold can be 5%.

[0168] If at least one first ratio is greater than or equal to the first threshold, the selection range of the selected region can be adjusted based on the fourth pixel set. Specifically, if at least one first ratio is greater than or equal to the first threshold, the process can proceed to step S460.

[0169] If the first ratio meets the target condition, the process can proceed to step S480. The target condition may include a first condition or a second condition. The first condition includes that each of the first ratios is less than the third threshold. The second condition includes that each of the first ratios is less than the first threshold and the fourth pixel set includes at least one first pixel set, and the third threshold is less than the first threshold.

[0170] Furthermore, if all first ratios are less than the first threshold and all are greater than the third threshold, and the fourth pixel set includes at least one first pixel set, the process can jump back to step S440. In some embodiments, a prompt message can be generated first, and then the process can return to step S440. The prompt message can be used to prompt the user to redefine the expanded selection area.

[0171] Step S460: Significance segmentation.

[0172] Specifically, a saliency segmentation network can be used to refine the edge pixels in the fourth pixel set and find the sixth pixel set that has an attachment relationship with the fifth pixel set, thereby adjusting the selection range of the bounding box area to the region corresponding to the refined fourth pixel set and the sixth pixel set.

[0173] Step S470: Obtain the target segmentation mask, edge contour lines, and label information.

[0174] Therefore, the target image region in the image to be recognized that matches the adjusted bounding box can be determined. Specifically, the adjusted bounding box can determine the set of pixels included, which can also be called the target segmentation mask. In addition, the adjusted bounding box can also determine the edge contour line, which can be the envelope line formed by the edge pixels in each pixel set in the bounding box. Furthermore, the label information corresponding to each pixel set in the bounding box can also be obtained. Then, based on the target segmentation mask, edge contour line, and label information, the target image region is determined from the image to be recognized, improving the accuracy of determining the target image region.

[0175] Step S480: Interactive segmentation.

[0176] If the first ratio meets the target condition, the target object matching the selected region can be identified from the image to be identified through interactive segmentation, and then the image region corresponding to the target object is taken as the target image region.

[0177] Optionally, after identifying the target object matching the selected region from the image to be identified through interactive segmentation, step S460 can be executed to refine the pixels at the edge of the pixel set corresponding to the target object, thereby using the image region corresponding to the refined target object as the target image region.

[0178] Optionally, the interactive segmentation can be performed locally on the electronic device or on a cloud server; this application embodiment does not impose specific limitations.

[0179] For a detailed description of each of the above steps, please refer to the foregoing embodiments; they will not be repeated here.

[0180] The image region determination method provided in this application effectively avoids the shortcomings of interactive segmentation (lacking label information) and panoramic segmentation (being limited to a limited number of categories) by combining interactive segmentation and panoramic segmentation. For common objects, panoramic segmentation is used to obtain their segmentation mask information and label information; for some uncommon objects, interactive segmentation is used for segmentation. In addition, by combining crowd segmentation algorithms and shadow detection and segmentation algorithms, the method achieves the goal of intelligently selecting all objects and obtaining label information for common objects.

[0181] Through experiments, the inventors verified that the image region determination method provided in this application achieves a mean intersection-over-union (MLOU) of 70.28 and a panoptic quality (PQ) of 71.84 in panoramic segmentation, demonstrating excellent performance in both recognition and segmentation. Furthermore, the image region determination method provided in this application achieves a mean MLOU of 89.61 in interactive segmentation, with a boundary residual rate reduced to 4.89%, indicating that the interactive segmentation model in this application performs well in both accuracy and boundary handling.

[0182] In addition, the inventors compared the image region determination method provided in this application with traditional methods for simple and complex scenarios, and scored different items, resulting in Tables 1 and 2 below.

[0183] Table 1

[0184]

[0185] Table 2

[0186]

[0187] As can be seen, whether for the simple scene in Table 1 or the complex scene in Table 2, the image region determination method provided in this application outperforms the traditional method in all scores.

[0188] Furthermore, the inventors, in conjunction with the acquired images and using the image region determination method provided in this application, determined the target image region and then performed a deletion process on the target image region. Detailed results can be found in [the relevant documentation / document / reference]. Figure 7 , Figure 8 as well as Figure 9 .

[0189] in, Figure 7 This diagram illustrates the effect of eliminating a target image region determined by the image region determination method provided in this application. Specifically, Figure 7 The image to be identified is shown as 710. The bounding box region determined by the target user through a bounding box operation on the image to be identified 710 is the bounding box region 720. Then, the target image region 730 is obtained by the image region determination method provided in this application embodiment. Thus, after the target image region 730 is eliminated, the eliminated image 740 is obtained.

[0190] It can be seen that, Figure 7 In the target image region 730, objects that have an attachment relationship with the target object are also intelligently added to the selection area.

[0191] For example, please refer to Figure 8 .in, Figure 8 This diagram illustrates the effect of eliminating a target image region determined by the image region determination method provided in this application. Specifically, Figure 8 The image to be identified is shown as 810. The bounding box region determined by the target user through a bounding box operation on the image to be identified 810 is the bounding box region 820. Then, the target image region 830 is obtained by the image region determination method provided in this application embodiment. Thus, after the target image region 830 is eliminated, the eliminated image 840 is obtained.

[0192] It can be seen that, in Figure 8 Within the target image region 830, crowds can be intelligently added to the selected area.

[0193] For example, please refer to Figure 9 .in, Figure 9 This diagram illustrates the effect of eliminating a target image region determined by the image region determination method provided in this application. Figure 9The image to be identified is shown as 910. The bounding box region determined by the target user through a bounding box operation on the image to be identified 910 is the bounding box region 920. Then, the target image region 930 is obtained by the image region determination method provided in this application embodiment. Thus, after the target image region 930 is eliminated, the eliminated image 940 is obtained.

[0194] It can be seen that, in Figure 9 Within the target image region 930, the human figure's shadow can be intelligently added to the selected area.

[0195] Furthermore, the inventors discovered that individual optimization can be performed on images to be recognized in specific scenarios, such as images where multiple people or objects overlap. This improves the accuracy of acquiring the target image region for the image to be recognized in that specific scenario.

[0196] Please see Figure 10 , Figure 10 The diagram shows a structural block diagram of an image region determination device provided in an embodiment of this application. The image region determination device 1000 includes: a first acquisition unit 1010, a second acquisition unit 1020, a third acquisition unit 1030, a generation unit 1040, and a target image region determination unit 1050.

[0197] The first acquisition unit 1010 is used to acquire the image to be recognized and the bounding area determined by the target user by performing a bounding operation on the image to be recognized.

[0198] The second acquisition unit 1020 is used to acquire the first set of pixels corresponding to the crowd objects in the image to be identified.

[0199] The third acquisition unit 1030 is used to acquire the second set of pixels corresponding to the human portrait shadow in the image to be identified.

[0200] The generation unit 1040 is used to obtain a third set of pixels based on the first set of pixels and the second set of pixels.

[0201] The generation unit 1040 can also be used to perform object segmentation on the image to be identified to obtain a set of seventh pixels in the image to be identified to represent different objects; update the objects corresponding to the seventh pixels in the set of seventh pixels that match the shape of the first set of pixels to crowd objects; update the objects corresponding to the pixels in the set of seventh pixels that match the shape of the second set of pixels to portrait shadows; and use the updated set of seventh pixels as the third set of pixels.

[0202] The target image region determination unit 1050 is used to adjust the selection range of the framed region based on the third pixel point set, and determine the target image region in the image to be identified that matches the adjusted framed region.

[0203] Optionally, the target image region determination unit 1050 can also be used to search for a first set of pixels and a second set of pixels in the third set of pixels where at least some pixels are within the selected region, as a fourth set of pixels; adjust the selection range of the selected region based on the fourth set of pixels, and determine the target image region in the image to be identified that matches the adjusted selected region.

[0204] Optionally, the target image region determination unit 1050 can also be used to obtain a first ratio corresponding to each of the fourth pixel point sets, wherein the first ratio is the ratio between the number of pixels in the fourth pixel point set that are within the selected area and the total number of pixels in the fourth pixel point set; if at least one of the first ratios is greater than or equal to a first threshold, obtain the sum of the number of pixels in each of the fourth pixel point sets to obtain a first value; obtain the total number of pixels corresponding to the selected area as a second value; adjust the selection range of the selected area based on the second ratio of the first value and the second value; and determine the target image region in the image to be identified that matches the adjusted selected area.

[0205] Optionally, the target image region determination unit 1050 can also be used to adjust the selection range of the selected region to the region corresponding to the fourth pixel point set when the second ratio is greater than or equal to the second threshold.

[0206] Optionally, the target image region determination unit 1050 can also be used to refine the pixels at the edge from the fourth pixel set through a saliency segmentation network when the second ratio is greater than or equal to the second threshold; and adjust the selection range of the selected region to the region corresponding to the fourth pixel set after the refinement.

[0207] Optionally, the target image region determination unit 1050 can also be used to determine the fifth pixel set corresponding to a specified part of the person object in the fourth pixel set; find the sixth pixel set that has an attachment relationship with the fifth pixel set based on the image to be identified; and adjust the selection range of the boxed region to the region corresponding to the fourth pixel set and the sixth pixel set.

[0208] Optionally, the target image region determination unit 1050 can also be used to identify a target object matching the selected region from the image to be identified through interactive segmentation when the first ratio satisfies the target condition. The target condition includes a first condition or a second condition. The first condition includes that each first ratio is less than the third threshold. The second condition includes that each first ratio is less than the first threshold and the fourth pixel set includes at least one first pixel set. The third threshold is less than the first threshold. The image region corresponding to the target object is taken as the target image region.

[0209] Optionally, the target image region determination unit 1050 can also be used to generate prompt information and return to execute the acquisition of the bounding area determined by the target user for the image to be identified and subsequent steps when each of the first ratios is less than the first threshold and greater than the third threshold, and the fourth pixel set includes at least one first pixel set. The prompt information is used to prompt the user to redetermine the expanded bounding area, and the third threshold is less than the first threshold.

[0210] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0211] In the several embodiments provided in this application, the coupling between the units can be electrical, mechanical, or other forms of coupling. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0212] Please see Figure 11 , Figure 11 This illustration shows a structural block diagram of an electronic device according to an embodiment of this application. The electronic device 110 may be a smartphone, desktop computer, in-vehicle computer, server, or tablet computer, etc. The electronic device 110 in this application may include one or more of the following components: a processor 111, a memory 112, and one or more application programs, wherein the processor 111 is electrically connected to the memory 112, and the one or more programs are configured to perform the methods described in the foregoing embodiments.

[0213] Processor 111 may include one or more processing cores. Processor 111 connects to various parts within the electronic device 110 using various interfaces and lines, and performs various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets, or instruction sets stored in memory 112, and by calling data stored in memory 112. Optionally, processor 111 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 111 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 111 and may be implemented separately using a communication chip. Specifically, the methods described in the foregoing embodiments can be executed by one or more processors 111.

[0214] In some implementations, memory 112 may include random access memory (RAM) or read-only memory (ROM). Memory 112 can be used to store instructions, programs, code, code sets, or instruction sets. Memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created by the electronic device 110 during use.

[0215] Please see Figure 12 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 1200 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0216] The computer-readable storage medium 1200 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1200 includes a non-volatile computer-readable storage medium. The computer-readable storage medium 1200 has storage space for program code 1210 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 1210 may be compressed, for example, in a suitable form.

[0217] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining an image region, characterized in that, include: Acquire the image to be recognized and the bounding area determined by the target user through a bounding operation on the image to be recognized; Obtain the first set of pixels corresponding to the crowd objects in the image to be identified; Obtain the set of second pixels corresponding to the human figure shadow in the image to be identified; A third set of pixels is obtained based on the first set of pixels and the second set of pixels; The selection range of the framed region is adjusted based on the third set of pixels to determine the target image region in the image to be identified that matches the adjusted framed region.

2. The method according to claim 1, characterized in that, The crowd object is at least one, and each crowd object corresponds to a first set of pixels. The portrait shadow is at least one, and each portrait shadow corresponds to a second set of pixels. The step of adjusting the selection range of the bounding box region based on the third set of pixels to determine the target image region in the image to be recognized that matches the adjusted bounding box region includes: In the third set of pixels, find the first set of pixels and the second set of pixels where at least some pixels are within the selected area, and use them as the fourth set of pixels; The selection range of the framed region is adjusted based on the fourth pixel set to determine the target image region in the image to be identified that matches the adjusted framed region.

3. The method according to claim 2, characterized in that, The step of adjusting the selection range of the bounding box region based on the fourth pixel set, and determining the target image region in the image to be recognized that matches the adjusted bounding box region, includes: Obtain a first ratio for each set of fourth pixels, wherein the first ratio is the ratio between the number of pixels in the fourth pixel set that are within the selected area and the total number of pixels in the fourth pixel set; If at least one of the first ratios is greater than or equal to the first threshold, the sum of the number of pixels in each of the fourth pixel sets is obtained to get the first value; The total number of pixels corresponding to the selected area is obtained as the second value. Based on the second ratio of the first value to the second value, the selection range of the selected area is adjusted; Determine the target image region in the image to be identified that matches the adjusted bounding box region.

4. The method according to claim 3, characterized in that, The adjustment of the selection range of the selected area based on the second ratio of the first value and the second value includes: If the second ratio is greater than or equal to the second threshold, the selection range of the selected area is adjusted to the area corresponding to the fourth pixel set.

5. The method according to claim 4, characterized in that, When the second ratio is greater than or equal to the second threshold, adjusting the selection range of the selected area to the area corresponding to the fourth pixel set includes: If the second ratio is greater than or equal to the second threshold, the pixels at the edge are refined from the fourth pixel set using a saliency segmentation network. The selection area of ​​the box is adjusted to the region corresponding to the fourth pixel set after fine processing.

6. The method according to claim 4, characterized in that, When the second ratio is greater than or equal to the second threshold, adjusting the selection range of the selected region to the region corresponding to the set of pixels includes: Determine the set of fifth pixels corresponding to a specified part of the human object in the set of fourth pixels; Based on the image to be identified, find the set of sixth pixels that have an attachment relationship with the set of fifth pixels; The selection area of ​​the box is adjusted to the area corresponding to the fourth pixel set and the sixth pixel set.

7. The method according to claim 3, characterized in that, The method further includes: When the first ratio satisfies the target condition, the target object matching the selected region is identified from the image to be identified through interactive segmentation. The target condition includes a first condition or a second condition. The first condition includes that each first ratio is less than the third threshold. The second condition includes that each first ratio is less than the first threshold and the fourth pixel set includes at least one first pixel set. The third threshold is less than the first threshold. The image region corresponding to the target object is taken as the target image region.

8. The method according to claim 3, characterized in that, The method further includes: If each of the first ratios is less than the first threshold and greater than the third threshold, and the fourth pixel set includes at least one first pixel set, a prompt message is generated and the process returns to obtain the bounding area determined by the target user's bounding operation on the image to be identified, as well as subsequent steps. The prompt message is used to prompt the user to redefine the expanded bounding area, and the third threshold is less than the first threshold.

9. The method according to claim 1, characterized in that, The process of obtaining a third set of pixels based on the first set of pixels and the second set of pixels includes: The image to be identified is segmented to obtain a set of seventh pixels in the image to be identified that represent different objects; The object corresponding to the seventh pixel in the seventh pixel set that matches the shape of the first pixel set is updated to a crowd object, and the object corresponding to the pixel in the seventh pixel set that matches the shape of the second pixel set is updated to a portrait shadow. The updated set of seventh pixels is used as the set of third pixels.

10. An image region determination device, characterized in that, include: The first acquisition unit is used to acquire the image to be recognized and the bounding area determined by the target user by performing a bounding operation on the image to be recognized. The second acquisition unit is used to acquire the first set of pixels corresponding to the crowd objects in the image to be identified; The third acquisition unit is used to acquire the set of second pixel points corresponding to the human portrait shadow in the image to be identified; A generation unit is configured to obtain a third set of pixels based on the first set of pixels and the second set of pixels; The target image region determination unit is used to adjust the selection range of the selected region based on the third pixel point set, and determine the target image region in the image to be identified that matches the adjusted selected region.

11. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-9.

Images