Obstruction determination method, electronic device, storage medium, and program product

Abstract

This application provides an occlusion determination method, electronic device, storage medium, and program product. The method mainly performs foreground and background image segmentation on the first image, and then performs a first segmentation process on the first image. In the first segmented image, an object repair function is automatically executed on the occluded first object, so that the third region corresponding to the first object can be fully represented in the first segmented image. Based on the intersection between the second region containing only the second object and the first region containing only the first object, it is finally determined whether the second object occludes the first object, which can improve the accuracy of the occlusion determination result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an occlusion determination method, electronic device, storage medium, and program product. Background Technology

[0002] With the development of image processing technology, after acquiring an image, image processing can be performed on specified objects in the image. For example, the state of a specified object (open eyes or closed eyes) can be identified. If a specified object in the image is occluded by other objects, the occlusion of other objects can be judged before image processing is performed on the specified object.

[0003] In related technologies, various visual features (such as texture, color, shape, etc.) are extracted from the image to annotate all objects in the image, and the annotation results of all objects are obtained. The occlusion of a specified object is then determined based on the annotation results.

[0004] However, because there are too many object categories in an image, labeling all objects in the image would result in excessively long image processing time and reduce image processing efficiency. Summary of the Invention

[0005] This application provides an occlusion determination method, electronic device, and storage medium, which can perform two different image segmentation processes on a first image to obtain different image segmentation results. Based on the region intersection, it determines whether a second object in the first image occludes a first object, thereby improving the accuracy of occlusion determination.

[0006] In a first aspect, an occlusion determination method is provided, the method comprising: acquiring a first image, the first image including a first object and a second object, the first object being located in a first region and the second object being located in a second region; performing a first image segmentation process on the first image to obtain a first segmented image, the first segmented image including a third region corresponding to the first image, the third region including the first region; and, in the case that the first object is occluded by the second object, performing object restoration processing on the first object to obtain a fourth region; and determining whether the second object occludes the first object based on the region intersection between the second region and the third region, the region intersection being used to indicate the overlapping region formed after the bounding box of the second region overlaps with the bounding box of the third region.

[0007] In the technical solution of this application, after acquiring a first image containing a first object and a second object, and with the first object and the second object respectively located in a first region and a second region, the first image is subjected to a first segmentation process to obtain a first segmented image. The first segmented image displays a third region corresponding to the first image. This third region includes not only the first region but also a fourth region after object restoration processing for cases where the first object is occluded. Whether the second object occludes the first object is determined based on the intersection of the second and third regions. That is, in addition to segmenting the first image into foreground and background images, the first image is further segmented. An object restoration function is automatically performed on the occluded first object in the first segmented image, ensuring that the third region corresponding to the first object is fully represented in the first segmented image. Therefore, based on the intersection of the second region containing only the second object and the first region containing only the first object, the accuracy of the occlusion judgment result is ultimately determined, improving the accuracy of the occlusion judgment result.

[0008] It should be understood that the first object and the second object in the first image are only used to distinguish different objects. Depending on the selected object, the corresponding occlusion situation is also different. For example, if the first object is object a and the second object is a fan, if object a is taken as the reference, then when the fan occludes object a's face, it can be considered that the second object occludes the first object. If the fan is taken as the reference, then the fan occludes object a's face, but object a's face does not occlude the fan. However, object a's hand occludes the fan because it is holding the fan handle.

[0009] In this embodiment, the first object is used as a reference to determine whether the second object occludes the first object.

[0010] Schematic, in the first image, the first object is located in the first region and the second object is located in the second region, wherein the first region and the second region are regions that do not intersect in the first image.

[0011] Optionally, the first object may be entirely located in the second region, or a portion of the first object may be located in the first region and a portion in the second region (in which case the portion is occluded by the second object). In this embodiment, the first and second regions are determined solely based on the display of the first and second objects in the first image. That is, if region a in the first object is occluded by the second object, then region a will belong to the second region.

[0012] Indicatively, the third region includes the first region and the fourth region. As mentioned above, if region a in the first object is occluded by the second object, then during the first image segmentation process, if region a is identified as being occluded by the second object, the occlusion of region a by the second object is ignored. The other regions in the first object excluding region a are taken as the first region, region a is taken as the fourth region, and the first region and the fourth region are taken as the third region.

[0013] It should be understood that the third region is used to indicate the complete area in the first image where the first object is not occluded. The first region may be the same as the third region, or the first region may be smaller than the third region (that is, in this case, a part of the first object is occluded by the second object).

[0014] Indicatively, the region intersection refers to the overlap between the bounding boxes of the second and third regions. If there is an overlap between the bounding boxes of the second and third regions, it means that the second object occludes the first object. If there is no overlap between the bounding boxes of the second and third regions, it means that the second object does not occlude the first object.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a reference detection box for the first object in the first image, the reference detection box indicating the position of the first object in the first image; obtaining a first detection box for the second object in the first image based on the region intersection between the second and third regions, the first detection box indicating the position of the second object in the first image; and determining whether the second object occludes the first object based on the overlap between the first detection box and the reference detection box. By setting a reference detection box for the first object in the first image and obtaining the first detection box for the second object in the first image based on the region overlap between the second and third regions, the edge overlap between the reference detection box and the first detection box is calculated, thereby improving the accuracy of occlusion detection.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: when there is an overlapping area between the first detection box and the reference detection box, determining the occlusion area caused by the second object on the first object based on the area of ​​the overlapping area. Through this method, even when it is determined that the second object occludes the first object, the occlusion area caused by the second object on the first object can be determined based on the area of ​​the overlapping area between the first detection box and the reference detection box. Besides determining whether occlusion occurs, it can also determine the occlusion area, further improving the accuracy of the occlusion determination result.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: performing second image segmentation on the first image to obtain a second segmented image, the second segmented image including a first region and a second region; performing morphological erosion on the second segmented image to obtain an eroded image, the eroded image including a fifth region, the fifth region including a second object, the area of ​​the second object in the eroded image being smaller than the area of ​​the second object in the first image; obtaining a first detection region based on the intersection between the fifth region and the third region, the first detection region being used to indicate the position of the second object in the eroded image; performing morphological dilation on the first detection region to obtain a dilated image; performing object filtering on the second object in the dilated image to obtain filtered objects, the filtered objects being used to indicate the second object occluding the first object; obtaining a first detection box based on the bounding rectangle corresponding to the filtered object. By performing the above methods, morphological operations (including erosion and dilation operations) are run to accurately locate the position of the second object in the first image, improving the accuracy of the second object's location.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a second detection region based on the region intersection between the dilated image and the third region, the second detection region being used to indicate the position of the filtering object in the dilated image; and selecting the second object located in the second detection region as the filtering object. By determining the filtering object based on the region intersection between the dilated image and the second region in the above manner, second objects that do not obstruct the region where the first object is located can be filtered out, thus improving the accuracy of the second object identification.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: performing a first image segmentation process on the first image using a first segmentation model to obtain a first segmented image; and performing a second image segmentation process on the first image using a second segmentation model to obtain a second segmented image. By using different segmentation models to perform different image segmentation processes on the first image, two segmented images with different regions are obtained, improving the accuracy of the image segmentation results. Furthermore, by using two models to perform different image segmentation operations on the first image, the complex task can be simplified into two lightweight tasks, each executed by a different image segmentation model, further improving image processing efficiency and reducing latency.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, the above method further includes: acquiring a first sample image, which includes a first sample object and a second sample object, wherein the second sample object occludes a first sample region of the first sample object; performing object annotation on the second sample object to obtain object annotation results of the second sample object in the first sample image, wherein the object annotation results include at least one of the object category, position, bounding box, or first mask region of the second sample object in the first sample image; and training the second sample model based on the object annotation results and the first sample image to obtain a second segmentation model. By training the second sample model based on automatically labeled training data in the above method, the training efficiency and accuracy of the model can be greatly improved.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: performing category labeling on the second sample object to obtain the object category corresponding to the second sample object; performing detection processing on the first sample image based on the object category to obtain a sample detection box of the second sample object in the first sample image, the sample detection box being used to indicate the position of the second sample object in the first sample image; performing image segmentation processing on the first sample image based on the sample detection box to obtain a first mask region of the second sample object in the first sample image; and using the object category, sample detection box, and first mask region as the object labeling result. By utilizing different labeling methods, including category labeling, detection box labeling, and mask labeling, the labeling task of the training data is decomposed into multiple different lightweight tasks for separate execution, which can reduce the difficulty of the labeling task and further improve the efficiency and accuracy of model training.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining a reference category related to the first sample object; removing the reference category from the object categories to obtain a filtered category; and performing detection processing on the first sample image based on the filtered category to obtain a sample detection box of the second sample object in the first sample image. By removing the reference category related to the first object from the object categories after category labeling, the method ensures that the final filtered categories all belong to the category corresponding to the occluded object, improving the accuracy of category localization and further enhancing the model training accuracy.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, the above method further includes: acquiring a second sample image, the second sample image including a third sample object; performing image segmentation processing on the third sample image to obtain a second mask region of the third sample object in the second sample image; and training a first sample model based on the second mask region, object annotation results, and the second sample image to obtain a first segmentation model.

[0024] In a second aspect, an occlusion determination device is provided, the device comprising a unit consisting of software and / or hardware for performing any of the methods of the first aspect.

[0025] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it is able to implement any of the methods of the first aspect.

[0026] Fourthly, a chip is provided, including a processor for reading and executing a computer program stored in a memory, wherein the computer program, when executed by the processor, is capable of implementing any of the methods of the first aspect.

[0027] Optionally, the chip also includes a memory electrically connected to the processor.

[0028] Optionally, the chip may also include a communication interface.

[0029] Fifthly, a computer-readable storage medium is provided that stores a computer program that, when executed by a processor, can implement any of the methods of the first aspect.

[0030] In a sixth aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, can implement any of the methods of the first aspect. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating an occlusion determination according to an embodiment of this application.

[0032] Figure 2 This is a schematic diagram of an occlusion determination method in an image editing scenario according to an embodiment of this application.

[0033] Figure 3 This is a flowchart of an occlusion determination method according to an embodiment of this application.

[0034] Figure 4 This is a schematic diagram of a first segmented image according to an embodiment of this application.

[0035] Figure 5 This is a schematic diagram of a second segmented image according to an embodiment of this application.

[0036] Figure 6 This is a schematic diagram of a first detection area determination process according to an embodiment of this application.

[0037] Figure 7 This is a schematic diagram of a second detection area acquisition process according to an embodiment of this application.

[0038] Figure 8 This is a flowchart of a segmentation model training method according to an embodiment of this application.

[0039] Figure 9 This is a flowchart of an object annotation result generation process according to an embodiment of this application.

[0040] Figure 10 This is a schematic diagram of a mask area annotation display according to an embodiment of this application.

[0041] Figure 11 This is a schematic flowchart of an occlusion determination method according to an embodiment of this application.

[0042] Figure 12 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0043] The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0044] In related approaches, various visual features (such as texture, color, and shape) are extracted from the image to annotate all objects, resulting in annotations for all objects. The occlusion status of a specific object is then determined based on these annotations. However, because images contain a large number of object categories, annotating all objects in the image leads to excessively long image processing times and reduces processing efficiency.

[0045] Therefore, how to accurately determine the occlusion situation is a key issue.

[0046] In view of this, this application provides an image processing method. For ease of understanding, a specific example is given below. Please refer to the illustrative example. Figure 1 It illustrates an occlusion determination diagram provided by an exemplary embodiment of this application, such as... Figure 1 As shown, the first image is currently displayed, which includes a first object and a second object. The first object is located in region 110, and the second object is located in region 120. Figure 1 It can be seen that the area 120 where the second object is located overlaps with the actual area where the first object is located. In other words, area 110 is used to represent the area where the first object is displayed in the first image.

[0047] After performing a first image segmentation process on the first image, a first segmented image 100 is obtained. The first segmented image 100 includes region 110 and region 130. Region 130 is identified as the region belonging to the first object in the first segmented image 100. Therefore, the actual region corresponding to the first object in the first segmented image 100 is the sum of the regions of region 110 and region 130.

[0048] It is worth noting that region 130 and region 120 are the same region. In some other embodiments, region 130 is smaller than region 120, but this application does not limit this.

[0049] Based on the intersection of region 120 and region , determine whether the second object in the first image occludes the first object. Figure 1 As shown, the second object occludes the first object, and the occlusion area is region 140.

[0050] Below, we will provide a detailed explanation of each photo editing scenario. Please refer to the following: Figure 2 This illustration shows a schematic diagram of an occlusion determination method in an image retouching scenario provided by an exemplary embodiment of this application, such as... Figure 2 As shown, during the operation of a gallery application on an electronic device, a gallery interface 210 is displayed. The gallery interface 210 displays multiple images, including a first image 201 and a second image 202. Both the first image 201 and the second image 202 display object A. In another scenario, the first image 201 is displayed in the gallery interface 210, but the second image 202 is stored in the gallery application but not displayed in the gallery interface 210.

[0051] When the first image 201 is selected, the image display interface 220 is displayed, showing the first image 201. Additionally, the image display interface 220 also displays an editing control 21. When a trigger operation is received on the editing control 21, the editing interface 230 corresponding to the first image 201 is displayed. At this point, under feasible conditions, an occlusion judgment is performed on the first image 201 to determine whether object A in the first image 201 is occluded by other objects. Specifically, this can be done according to... Figure 1 If object A is occluded by other objects, the occlusion determination method described above will display "Current image cannot be repaired" in the editing interface 230. Figure 2 (Not shown in the image). In addition, the above occlusion determination method is also performed on the second image 202, in addition to the first image 201. If the occlusion determination result of the second image 202 is that object A is occluded by other objects, then the second image 202 is not used as the reference image of the first image 201.

[0052] If the occlusion determination result corresponding to the first image 201 indicates that object A is not occluded by other objects, a repair control 22 is displayed in the editing interface 230. When a trigger operation is received on the repair control 22, a repair editing interface 240 is displayed, which displays a closed-eye repair control 23, used to repair object A, which is in a closed-eye state, to an open-eye state. Upon receiving a trigger operation on the closed-eye repair control 23, under feasible conditions, an occlusion determination is performed on the first image 201 to determine whether object A in the first image 201 is occluded by other objects. Specifically, this can be done according to... Figure 1 If object A is occluded by other objects, the occlusion determination method described above will display "Current image cannot be repaired" in the editing interface 230. Figure 2 (Not shown in the image). If the eyes of object A are identified as not being obscured by other objects, the state of the eyes of object A in the first image 201 is repaired to obtain the third image 203, in which object A is shown with its eyes open.

[0053] That is, according to Figure 2 Based on the above content, in the image editing scenario, the process of judging occlusions in the selected image can be executed at multiple times during the entire image processing process. This application does not specifically limit the timing of the occlusion judgment.

[0054] Optionally, the above description only refers to adjusting the state of one object in the image. In this embodiment, the state of the facial features of a single object in the same image can be adjusted, or the state of the facial features of multiple objects in the same image can be adjusted. In this case, the same second image (in which case, all objects contained in the first image exist in the second image) can be used to adjust the state of the facial features of each object in the first image. Alternatively, different second images can be used for different objects in the first image (for example, if the first image includes object 1 and object 2, image 1 containing object 1 can be used as the second image corresponding to object 1 in the first image, and image 2 containing object 2 can be used as the second image corresponding to object 2 in the first image) to adjust the state of the facial features of each object in the first image.

[0055] Besides image editing, occlusion detection can also be applied to vehicle occlusion detection. While the vehicle is in motion, when a camera on the vehicle captures a view of the road, this view is used as the first image. Taking a first image containing vehicle a and road b as an example, the task is to determine whether vehicle a occludes a bend in road b. Figure 1If the identification method identifies that vehicle a is obscuring the bend at the end of road b, the obscured image is further repaired to obtain a final repaired image in which road b is not obscured, making it easier for users to familiarize themselves with the road conditions.

[0056] Before detailing the methods provided in the embodiments of this application, the execution entities provided in the embodiments of this application will be introduced first. In one case, the methods provided in the embodiments of this application can be executed by an electronic device with image capturing capabilities. The main application scenario is a scenario where a photograph is taken using an electronic device, specifically, it is applied to an image processing scenario of an electronic device. As an example and not a limitation, the electronic device can be, but is not limited to, terminals such as mobile phones, smartwatches, and tablets, and the embodiments of this application do not limit this. In another case, the methods provided in the embodiments of this application can be executed collaboratively by an electronic device with image capturing capabilities and a server. More specifically, the technical solutions in the embodiments of this application can be applied to a terminal device and a server.

[0057] The electronic devices (i.e., the terminal devices mentioned above) in this application embodiment can be televisions, desktop computers, laptops, or portable electronic devices such as mobile phones, tablets, cameras, camcorders, video recorders, or other electronic devices with photography functions, electronic devices in 5G networks, or electronic devices in future evolved public land mobile networks (PLMNs), etc. This application does not limit them.

[0058] The server in this application embodiment can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. Alternatively, the server can also be implemented as a node in a blockchain system. Therefore, this application does not limit the implementation architecture of the server.

[0059] The occlusion determination method provided in this embodiment can be executed by the electronic device alone, by the server alone, or by the electronic device and the server in collaboration. This embodiment does not limit this.

[0060] It should be noted that the information (including but not limited to object device information, object personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0061] The reasoning process for the occlusion detection scheme will be explained in detail below.

[0062] This is illustrative; please refer to it. Figure 3 It illustrates a flowchart of an occlusion determination method provided in an exemplary embodiment of this application, such as... Figure 3 As shown, the method includes the following steps.

[0063] Step 310: Obtain the first image.

[0064] A first image is obtained, which includes a first object and a second object. Based on the first object, this embodiment determines whether the second object occludes the first object.

[0065] Step 320: Input the first segmentation model.

[0066] During the process of inputting the first image into the first segmentation model, the first segmentation model performs pixel prediction on the second region where the second object is located. If the prediction result is that the second object does not occlude the first object, the region where the second object is located is identified as a background region. If the prediction result is that the second object occludes the first object, the region corresponding to the second object is identified as a foreground region. At this time, the region where the first object is located is also identified as a foreground region. If the region where the second object is located includes region a and region b, where region a occludes the first object and region b is not occluded, then region a is identified as a foreground region and region b is identified as a background region. Through this first segmentation model, the region where the first object is occluded by the second object can also be identified as a foreground region of the same type as the region where the first object is located, thus ignoring the regional influence of the occlusion on the first object.

[0067] It is worth noting that for the first segmentation model, the focus is on the foreground region.

[0068] Step 330: Input the second segmentation model.

[0069] The first image is segmented using a second segmentation model. The area where the first object is displayed in the first image is designated as the foreground region, and all other regions besides the first object are identified as background regions. In other words, if the second object occludes the first object, and the occluded area is region c, then region c is identified as the background region. If the second object does not occlude the first object, and the area where the second object is located in the first image is region d, then region d is also identified as the background region.

[0070] That is, the second segmentation model is used to segment the foreground and background regions of the first image.

[0071] In other words, the difference between the first segmentation model and the second segmentation model is that the first segmentation model can specifically identify the occlusion of the first object by the second object as a foreground region or a background region, while in the second segmentation model, regardless of whether the second object occludes the first object, the area displayed by the second object in the first image is always a background region.

[0072] In this embodiment, the focus of the second segmentation model is the background region.

[0073] Step 340: Obtain the first segmented image.

[0074] Indicatively, when the first image is segmented using the first segmentation model, and the first object is occluded by the second object, the occluded area is the fourth region. The second object is located in the second region in the first image. Therefore, the first segmented image includes the first region where the first object is displayed in the first image, and the fourth region where the first object is occluded by the second object. The first region and the fourth region are taken as the third region, that is, the foreground region corresponding to the first image. The region in the first image other than the foreground region is taken as the background region.

[0075] This is illustrative; please refer to it. Figure 4 This illustrates a schematic diagram of a first segmented image provided in an exemplary embodiment of this application, such as... Figure 4 As shown, the first segmented image 400 is currently displayed, which includes a white region 420 and a black region 410. The white region 410 refers to the foreground region. If a second object occludes the first object in the foreground region, the occluded region is also identified as the foreground region. The black region 420 refers to the background region.

[0076] Step 350: Obtain the second segmented image.

[0077] To illustrate, after the first image is segmented using the second segmentation model, the area where the first object is displayed in the first image is designated as the foreground region, and the areas outside the foreground region are identified as the background region.

[0078] This is illustrative; please refer to it. Figure 5 This illustrates a schematic diagram of a second segmented image provided in an exemplary embodiment of this application, such as... Figure 5 As shown, the second segmented image 500 is currently displayed, which includes a white region 510 and a black region 520, wherein the white region 510 refers to the foreground region and the black region 520 refers to the background region.

[0079] Step 360: Obtain the corrosion results.

[0080] Morphological erosion is performed on the second segmented image to obtain the corresponding erosion result.

[0081] Morphological erosion is an operation based on structuring elements (also known as erosion kernels). A structuring element is a small shape template used for pixel-by-pixel comparison with the second segmented image. During erosion, a structuring element is placed at a pixel location in the second segmented image. If all pixels within the structuring element match the corresponding pixels in the image (typically in binary images, a match means both are 1 or both are 0, depending on the definition of erosion and the background value), the pixel remains unchanged; otherwise, it is set to the background value (usually 0). This process iterates through every pixel in the image until the entire second segmented image has been processed.

[0082] In this embodiment, the structuring element is a pre-defined value, for example: the kernel size (kernel_size) is 5. The elliptical structure of 5.

[0083] After obtaining the erosion result, the area displayed by the second object in the erosion result (i.e., the fifth region) will be slightly smaller than the area corresponding to the second object in the second segmented image. In this way, during the execution of step 370, the background area in the erosion result will not intersect with the foreground area in the second segmented image, and the actual position of the second object in the first image can be located more accurately.

[0084] Step 370: Determine the first detection area.

[0085] Indicatively, the intersection of the foreground region in the first segmented image and the background region in the erosion result is calculated, and the designated region that belongs to both the foreground region in the first segmented image and the background region in the dilation result is determined as the first detection region.

[0086] This is illustrative; please refer to it. Figure 6This illustration shows a schematic diagram of a first detection region determination process provided by an exemplary embodiment of this application. The first image is segmented using the first and second segmentation models described above to obtain a first segmented image and a second segmented image. The second segmented image is then subjected to morphological erosion to obtain an eroded image. Therefore, the eroded image includes a foreground region 610 and a background region 620 (the diagonal line region identifies the background region). The first segmented image includes a foreground region 630 (the diagonal line indicates the foreground region). Based on the intersection of the background region 620 and the foreground region 630, region 640 is determined as the first detection region. The first detection region is the region corresponding to the second object in the first image after morphological erosion.

[0087] Step 380: Obtain the dilated image.

[0088] Since the first detection region is the region corresponding to the second object in the first image after morphological erosion, it is also necessary to perform a morphological dilation operation on the image corresponding to the first detection region to restore the region corresponding to the second object in the first detection region to the actual region in the first image.

[0089] Step 390: Determine the second detection area.

[0090] After obtaining the dilated image, based on the region intersection between the dilated image and the first segmentation image, regions that do not belong to the region where the first object is located but are also identified as the first detection region can be removed, so that the filtered objects in the final second detection region all have an intersection with the region where the first object is located, which helps to remove other objects that do not obstruct the first object.

[0091] This is illustrative; please refer to it. Figure 7 The illustration shows a schematic diagram of the second detection region acquisition process provided by an exemplary embodiment of this application. After acquiring the dilated image, the dilated image includes multiple first detection regions, including region 701, region 702 and region 703. Based on the region intersection between the dilated image and the first segmentation image, it is finally determined that region 701 is a region that belongs to both the first detection region and has an intersection with the region where the first object is located. Therefore, region 701 is determined as the second detection region.

[0092] Step 3100: Obtain the first detection box.

[0093] After determining the second detection area, the smallest bounding rectangle corresponding to the second detection area is determined based on the area outline of the second detection area, and it is used as the first detection box.

[0094] Step 3110: Obtain the eye rectangle.

[0095] In this embodiment, after obtaining the first image, for the first object in the first image, based on the eye area of ​​the first object, it is determined whether the second object occludes the eye area of ​​the first object. Therefore, after obtaining the first image, it is necessary to crop the first image to obtain the eye rectangle corresponding to the first image.

[0096] Step 3120: Obtain the face rectangle.

[0097] In this embodiment, after obtaining the first image, for the first object in the first image, based on the face area of ​​the first object, it is determined whether the second object occludes the face area of ​​the first object. Therefore, after obtaining the first image, it is necessary to crop the first image to obtain the face rectangle corresponding to the first image.

[0098] Obtain the region intersection relationship between the face rectangle / eye rectangle and the first detection box. If there is an intersection between the face rectangle / eye rectangle and the first detection box, proceed to step 3130; otherwise, proceed to step 3140.

[0099] Step 3130, occlusion result.

[0100] If there is an intersection between the face rectangle / eye rectangle and the first detection box, it means that the second object occludes the face / eye area of ​​the first object.

[0101] Step 3140, Unobstructed Result.

[0102] If there is no intersection between the face rectangle / eye rectangle and the first detection box, it means that the second object does not occlude the face / eye area of ​​the first object.

[0103] In this embodiment, the intersection-union ratio (IUR) with the face rectangle / eye rectangle is calculated. If the IUR is greater than 0, the face region / eye region is considered to be occluded; if it is equal to 0, the face region / eye region is considered not to be occluded.

[0104] The training process for the occlusion scheme will be explained in detail below.

[0105] This is illustrative; please refer to it. Figure 8 The diagram illustrates a flowchart of a segmentation model training method provided in an exemplary embodiment of this application, which includes the following steps.

[0106] Step 810: Obtain the first sample image.

[0107] To illustrate, the first sample image is obtained from the training dataset. The first sample image includes a first sample image and a second sample image, wherein the second sample image occludes the first sample region corresponding to the first sample image.

[0108] Step 820, Object annotation.

[0109] Object annotation is performed on the second sample object in the first sample image to obtain the object annotation result of the second sample object in the first sample image.

[0110] Step 830: Obtain the object annotation results.

[0111] Optionally, the object annotation result includes at least one of the object category, location, bounding box, or first mask region of the second sample object in the first sample image.

[0112] The following section details the process of obtaining object annotation results.

[0113] This is illustrative; please refer to it. Figure 9 It illustrates a flowchart of the object annotation result generation process provided in an exemplary embodiment of this application, such as... Figure 9 As shown, the method includes the following steps.

[0114] Step 910: Obtain the first sample image.

[0115] To illustrate, the first sample image is obtained from the training dataset. The first sample image includes a first sample image and a second sample image, wherein the second sample image occludes the first sample region corresponding to the first sample image.

[0116] Step 920: Category labeling.

[0117] The second sample object is labeled with its category to obtain the object category corresponding to the second sample object.

[0118] In this embodiment, the recognize anything model (RAM model) is called to label all second sample objects that occlude the first sample object in the first sample image, so as to obtain the object category corresponding to each second sample object. If there are two identical second sample objects, a second sample object set is generated and the corresponding object category is labeled on the sample object set.

[0119] Step 930: Category removal.

[0120] After obtaining the object categories corresponding to multiple second sample objects, the reference categories related to the first object are first obtained: for example, eyes, nose, mouth, etc. If the above reference categories exist in multiple object categories, these reference categories are removed from the object categories.

[0121] Step 940: Filter categories.

[0122] If the above reference categories exist among multiple object categories, these reference categories will be removed from the object categories, and the remaining object categories will be used as the filtering categories.

[0123] Step 950: Obtain the sample detection frame.

[0124] The detection model (Grounding DINO) is used to detect bounding boxes for the filtering categories, and the bounding boxes of the second sample objects corresponding to the filtering categories in the first sample image are determined. The bounding boxes are used to determine the position of the second sample objects in the first sample image.

[0125] Step 960: Obtain the first mask area.

[0126] Input the sample detection box into the trained segmentation model (Segment Anything Model, SAM model) to generate the first mask region corresponding to the sample detection box.

[0127] Step 970, save.

[0128] Store the first mask region of the second sample object corresponding to the selected category as labeled data.

[0129] This is illustrative; please refer to it. Figure 10 It shows an exemplary mask area annotation display diagram of this application, such as Figure 10 As shown, the first sample image is currently displayed. The first sample image includes a first sample object and a second sample object that occludes the first sample object. Through the object annotation method described above, the first mask region 1010 corresponding to the second sample object is finally obtained, as well as the object category corresponding to the second sample object: pen.

[0130] Step 840: Obtain the second sample image.

[0131] To illustrate, the second sample image is obtained from the training dataset, and the second sample image includes the third sample object, but the third sample object is not occluded by other objects.

[0132] Step 850: Obtain the second mask area.

[0133] Schematic, the second mask region is used to indicate the region corresponding to a specified feature (e.g., facial feature) of the third sample object in the second sample image.

[0134] Optionally, the second mask region is data that has already been labeled in the training dataset; or, a pre-trained mask recognition model is used to perform mask recognition on the second sample image to determine the second mask region corresponding to the third sample object.

[0135] Step 860: The second segmentation model is trained.

[0136] The second sample model is trained based on the object annotation results and the first sample image to obtain the second segmentation model.

[0137] Step 870: The first segmentation model is trained.

[0138] The first sample model is trained based on the second mask region, the object annotation results corresponding to the first sample image, and the second sample image to finally obtain the first segmentation model.

[0139] It is worth noting that the training order of the first segmentation model and the second segmentation model can be as follows: the first segmentation model can be trained first, followed by the second segmentation model; the second segmentation model can be trained first, followed by the first segmentation model; or the first segmentation model and the second segmentation model can be trained simultaneously. This application does not limit the specific training order of the first and second segmentation models.

[0140] Figure 11 This is a schematic flowchart illustrating an occlusion determination method according to an embodiment of this application. The following is a description of... Figure 11 The steps shown will be explained. Figure 11 Can be used Figures 1-10 It can be executed by any electronic device in the system.

[0141] S1001, Obtain the first image.

[0142] The first image includes a first object and a second object, with the first object located in a first region and the second object located in a second region.

[0143] It should be understood that the first object and the second object in the first image are only used to distinguish different objects. Depending on the selected object, the corresponding occlusion situation is also different. For example, if the first object is object a and the second object is a fan, if object a is taken as the reference, then when the fan occludes object a's face, it can be considered that the second object occludes the first object. If the fan is taken as the reference, then the fan occludes object a's face, but object a's face does not occlude the fan. However, object a's hand occludes the fan because it is holding the fan handle.

[0144] In this embodiment, the first object is used as a reference to determine whether the second object occludes the first object.

[0145] S1002, perform first image segmentation processing on the first image to obtain a first segmented image, the first segmented image including the third region corresponding to the first image.

[0146] The third region includes the first region, and the fourth region obtained after object repair processing of the first object when the first object is occluded by the second object.

[0147] Schematic, in the first image, the first object is located in the first region and the second object is located in the second region, wherein the first region and the second region are regions that do not intersect in the first image.

[0148] Optionally, the first object may be entirely located in the second region, or a portion of the first object may be located in the first region and a portion in the second region (in which case the portion is occluded by the second object). In this embodiment, the first and second regions are determined solely based on the display of the first and second objects in the first image. That is, if region a in the first object is occluded by the second object, then region a will belong to the second region.

[0149] Indicatively, the third region includes the first region and the fourth region. As mentioned above, if region a in the first object is occluded by the second object, then during the first image segmentation process, if region a is identified as being occluded by the second object, the occlusion of region a by the second object is ignored. The other regions in the first object excluding region a are taken as the first region, region a is taken as the fourth region, and the first region and the fourth region are taken as the third region.

[0150] S1003, determine whether the second object occludes the first object based on the intersection of the second and third regions.

[0151] Among them, the region intersection is used to indicate the overlapping area formed after the boundary box of the second region overlaps with the boundary box of the third region.

[0152] It should be understood that the third region is used to indicate the complete area in the first image where the first object is not occluded. The first region may be the same as the third region, or the first region may be smaller than the third region (that is, in this case, a part of the first object is occluded by the second object).

[0153] Indicatively, the region intersection refers to the overlap between the bounding boxes of the second and third regions. If there is an overlap between the bounding boxes of the second and third regions, it means that the second object occludes the first object. If there is no overlap between the bounding boxes of the second and third regions, it means that the second object does not occlude the first object.

[0154] In some implementations, the method further includes: obtaining a reference detection box for the first object in the first image, the reference detection box indicating the position of the first object in the first image; obtaining a first detection box for the second object in the first image based on the region intersection between the second and third regions, the first detection box indicating the position of the second object in the first image; and determining whether the second object occludes the first object based on the overlap between the first detection box and the reference detection box. By setting a reference detection box for the first object in the first image and obtaining the first detection box for the second object in the first image based on the region overlap between the second and third regions, the edge overlap between the reference detection box and the first detection box is calculated, thereby improving the accuracy of occlusion detection.

[0155] In some implementations, the method further includes: when there is an overlapping area between the first detection box and the reference detection box, determining the occlusion area of ​​the second object on the first object based on the area of ​​the overlapping area. This method not only determines whether the second object occludes the first object, but also determines the occlusion area based on the area of ​​the overlapping area between the first and reference detection boxes. Therefore, in addition to determining whether occlusion occurs, it can also determine the occlusion area, further improving the accuracy of the occlusion determination result.

[0156] In some implementations, the above method further includes: performing second image segmentation on the first image to obtain a second segmented image, the second segmented image including a first region and a second region; performing morphological erosion on the second segmented image to obtain an eroded image, the eroded image including a fifth region, the fifth region including a second object, the area of ​​the second object in the eroded image being smaller than the area of ​​the second object in the first image; obtaining a first detection region based on the region intersection between the fifth region and the third region, the first detection region being used to indicate the position of the second object in the eroded image; performing morphological dilation on the first detection region to obtain a dilated image; performing object filtering on the second object in the dilated image to obtain filtered objects, the filtered objects being used to indicate the second object occluding the first object; obtaining a first detection box based on the bounding rectangle corresponding to the filtered object. By performing the above methods, morphological operations (including erosion and dilation operations) are run to accurately locate the position of the second object in the first image, improving the accuracy of the second object's location.

[0157] In some implementations, the method further includes: obtaining a second detection region based on the intersection between the dilated image and the third region, the second detection region indicating the position of the target object in the dilated image; and selecting the second object located in the second detection region as the target object. By determining the target object based on the intersection between the dilated image and the second region, the method can filter out second objects that do not obstruct the area where the first object is located, thus improving the accuracy of the second object identification.

[0158] In some implementations, the above method further includes: performing a first image segmentation process on the first image using a first segmentation model to obtain a first segmented image; and performing a second image segmentation process on the first image using a second segmentation model to obtain a second segmented image. By using different segmentation models to perform different image segmentation processes on the first image, two segmented images with different regions are obtained, improving the accuracy of the image segmentation results. Furthermore, by using two models to perform different image segmentation operations on the first image, the complex task can be simplified into two lightweight tasks, each executed by a different image segmentation model, further improving image processing efficiency and reducing latency.

[0159] In some implementations, the above method further includes: acquiring a first sample image, which includes a first sample object and a second sample object, wherein the second sample object occludes the first sample region of the first sample object; performing object annotation on the second sample object to obtain object annotation results of the second sample object in the first sample image, wherein the object annotation results include at least one of the object category, location, bounding box, or first mask region of the second sample object in the first sample image; and training the second sample model based on the object annotation results and the first sample image to obtain a second segmentation model. By training the second sample model based on automatically labeled training data in the above method, the training efficiency and accuracy of the model can be greatly improved.

[0160] In some implementations, the above method further includes: performing category labeling on the second sample object to obtain the object category corresponding to the second sample object; performing detection processing on the first sample image based on the object category to obtain a sample detection box of the second sample object in the first sample image, the sample detection box being used to indicate the position of the second sample object in the first sample image; performing image segmentation processing on the first sample image based on the sample detection box to obtain a first mask region of the second sample object in the first sample image; and using the object category, sample detection box, and first mask region as the object labeling result. By utilizing different labeling methods, including category labeling, detection box labeling, and mask labeling, the labeling task of training data is decomposed into multiple different lightweight tasks for separate execution, which can reduce the difficulty of the labeling task and further improve the efficiency and accuracy of model training.

[0161] In some implementations, the above method further includes: obtaining a reference category related to the first sample object; removing the reference category from the object categories to obtain a filtered category; and performing detection processing on the first sample image based on the filtered category to obtain a sample detection box of the second sample object in the first sample image. By removing the reference category related to the first object from the object categories after category labeling, the method ensures that the final filtered categories all belong to the category corresponding to the occluded object, improving the accuracy of category localization and further enhancing the model training accuracy.

[0162] In some implementations, the above method further includes: acquiring a second sample image, which includes a third sample object; performing image segmentation processing on the third sample image to obtain a second mask region of the third sample object in the second sample image; and training a first sample model based on the second mask region, object annotation results, and the second sample image to obtain a first segmentation model. In the technical solution of this application, after acquiring a first image containing a first object and a second object, and with the first object and the second object respectively located in a first region and a second region, a first segmentation processing is performed on the first image to obtain a first segmented image. The first segmented image displays a third region corresponding to the first image. The third region includes not only the first region but also a fourth region after object restoration processing is performed when the first object is occluded. Whether the second object occludes the first object is determined based on the region intersection between the second region and the third region. That is, in addition to segmenting the first image into foreground and background images, the first image will also undergo a first segmentation process. In the first segmented image, the object repair function will be automatically performed on the occluded first object, so that the third region corresponding to the first object can be fully represented in the first segmented image. Based on the intersection between the second region containing only the second object and the first region containing only the first object, it can be finally determined whether the second object occludes the first object, which can improve the accuracy of the occlusion judgment result.

[0163] The methods of the embodiments of this application have been described above with reference to the accompanying drawings. It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially, these steps are not necessarily executed in the order shown in the figures. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps. The apparatus of the embodiments of this application will now be described with reference to the accompanying drawings.

[0164] Figure 12 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Figure 12 As shown, the electronic device 1200 may include a processor 1210, an external memory interface 1220, an internal memory 1221, a universal serial bus (USB) interface 1230, a charging management module 1240, a power management module 1241, a battery 1242, an antenna 1, an antenna 2, a mobile communication module 1250, a wireless communication module 1260, an audio module 1270, a speaker 1270A, a receiver 1270B, a microphone 1270C, a headphone jack 1270D, a sensor module 1280, buttons 1290, a motor 991, an indicator 992, a camera 993, a display screen 994, and a subscriber identification module (SIM) card interface 995, etc.

[0165] The sensor module 1280 may include a pressure sensor 1280A, a gyroscope sensor 1280B, a barometric pressure sensor 1280C, a magnetic sensor 1280D, an accelerometer sensor 1280E, a distance sensor 1280F, a proximity sensor 1280G, a fingerprint sensor 1280H, a temperature sensor 1280J, a touch sensor 1280K, an ambient light sensor 1280L, a bone conduction sensor 1280M, etc.

[0166] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 1200. In other embodiments of this application, the electronic device 1200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0167] For example, Figure 12 The processor 1210 shown may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0168] The controller can serve as the central nervous system and command center of the electronic device 1200. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0169] In this embodiment, the processor 1210 is mainly used to launch the application loading page and respond to user input interactions.

[0170] The processor 1210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 1210 is a cache memory. This memory can store instructions or data that the processor 1210 has just used or that are used repeatedly. If the processor 1210 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 1210, and thus improves the efficiency of the system.

[0171] Electronic device 1200 implements display functions through a GPU, a display screen 994, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 994 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. Processor 1210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0172] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel. In some embodiments, the electronic device 1200 may include one or N display screens 994, where N is a positive integer greater than 1.

[0173] In this embodiment, the various interfaces are presented to the user primarily through the display screen 994, for example... Figure 1 The various interfaces within.

[0174] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0175] In this embodiment, the NPU can be used to identify the display content and instruct the processor 1210 to generate the display content, for example, it can instruct the processor 1210 to generate the display content according to... Figure 1 The content is generated and displayed in the form shown.

[0176] Pressure sensor 1280A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 1280A can be disposed on display screen 994. There are many types of pressure sensors 1280A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 1280A, the capacitance between the electrodes changes. Electronic device 1200 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 994, electronic device 1200 detects the intensity of the touch operation based on pressure sensor 1280A. Electronic device 1200 can also calculate the touch position based on the detection signal from pressure sensor 1280A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to the SMS application icon, a command to view SMS messages is executed. When a touch operation with a strength greater than or equal to the first pressure threshold is applied to the SMS application icon, the instruction to create a new SMS message is executed.

[0177] In this embodiment, the pressure sensor 1280A is mainly used to collect user interaction operations, such as swiping operations and clicking operations.

[0178] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0179] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0180] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0181] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0182] This application provides a computer program product, which includes a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0183] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0184] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0185] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0186] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0189] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0190] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0191] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0192] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0193] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An occlusion determination method, characterized in that, The method includes: Acquire a first image, which includes a first object and a second object, wherein the first object is located in a first region and the second object is located in a second region; The first image is subjected to a first image segmentation process to obtain a first segmented image. The first segmented image includes a third region corresponding to the first image. The third region includes the first region. A fourth region is obtained after object repair processing of the first object when the first object is occluded by the second object. Obtain a reference detection box of the first object in the first image, the reference detection box being used to indicate the position of the first object in the first image; The first image is subjected to a second image segmentation process to obtain a second segmented image, which includes the first region and the second region. The second segmented image is subjected to morphological erosion processing to obtain an eroded image. The eroded image includes a fifth region, and the fifth region includes the second object. The area of ​​the second object in the eroded image is smaller than the area of ​​the second object in the first image. Based on the intersection between the fifth region and the third region, a first detection region is obtained, which is used to indicate the position of the second object in the erosion image. The first detection region is subjected to morphological dilation processing to obtain a dilated image; The second object in the dilated image is filtered to obtain filtered objects, which are used to indicate the second object that occludes the first object; A first detection box is obtained based on the bounding rectangle corresponding to the filtered object. The first detection box is used to indicate the position of the second object in the first image. Based on the overlap between the first detection box and the reference detection box, it is determined whether the second object occludes the first object. The region intersection is used to indicate the overlapping area formed after the boundary box of the second region overlaps with the boundary box of the third region.

2. The method according to claim 1, characterized in that, The method further includes: In the case where there is an overlapping area between the first detection frame and the reference detection frame, the occlusion area caused by the second object on the first object is determined based on the area of ​​the overlapping area.

3. The method according to claim 1, characterized in that, The step of filtering the second object in the dilated image to obtain the filtered objects includes: Based on the intersection of the regions between the dilated image and the third region, a second detection region is obtained, which is used to indicate the position of the object to be screened in the dilated image. The second object located in the second detection area is used as the filtering object.

4. The method according to claim 1, characterized in that, The step of performing a first image segmentation process on the first image to obtain a first segmented image includes: The first image is obtained by performing a first image segmentation process on the first image using a first segmentation model; The step of performing a second image segmentation process on the first image to obtain a second segmented image includes: The first image is segmented using a second segmentation model to obtain the second segmented image.

5. The method according to claim 4, characterized in that, Before performing second image segmentation processing on the first image using the second segmentation model to obtain the second segmented image, the method further includes: A first sample image is obtained, which includes a first sample object and a second sample object, wherein the second sample object occludes the first sample region of the first sample object; The second sample object is annotated to obtain the object annotation result of the second sample object in the first sample image. The object annotation result includes at least one of the object category, position, bounding box, or first mask region of the second sample object in the first sample image. The second sample model is trained based on the object annotation results and the first sample image to obtain the second segmentation model.

6. The method according to claim 5, characterized in that, The step of annotating the second sample object to obtain the object annotation result of the second sample object in the first sample image includes: The second sample object is labeled with a category to obtain the object category corresponding to the second sample object. Based on the object category, the first sample image is detected to obtain a sample detection box of the second sample object in the first sample image. The sample detection box is used to indicate the position of the second sample object in the first sample image. Based on the sample detection box, the first sample image is segmented to obtain the first mask region of the second sample object in the first sample image; The object category, the sample detection box, and the first mask region are used as the object annotation result.

7. The method according to claim 6, characterized in that, The step of performing detection processing on the first sample image based on the object category to obtain the sample detection box of the second sample object in the first sample image includes: Obtain the reference category associated with the first sample object; The reference category is removed from the object category to obtain the filter category; Based on the filtering category, the first sample image is processed to obtain the sample detection box of the second sample object in the first sample image.

8. The method according to claim 5, characterized in that, Before performing the first image segmentation process on the first image using the first segmentation model to obtain the first segmented image, the method further includes: Obtain a second sample image, which includes a third sample object; The second sample image is segmented to obtain the second mask region of the third sample object in the second sample image. The first sample model is trained based on the second mask region, the object annotation result, and the second sample image to obtain the first segmentation model.

9. An electronic device, characterized in that, The electronic device includes a memory, one or more processors, and a computer program stored in the memory and executable on the processors, wherein the one or more processors execute the computer program to cause the electronic device to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by an electronic device, implements the method as described in any one of claims 1 to 8.

11. A computer program product, characterized in that, Includes a computer program that, when run, causes the method as described in any one of claims 1 to 8 to be performed.

Images