A method and device for detecting screen changes in an occlusion situation

Abstract

The application provides a method and device for detecting screen change under the condition of occlusion. First, a first image and a second image are acquired in a task scene. Then, screen regions are acquired in the first image and the second image, and perspective transformation is performed to obtain a first target screen image and a second target screen image. Next, a segmentation algorithm is used to segment the occluded regions and the non-occluded regions of the first target screen image and the second target screen image. Finally, a plurality of grids are added in the first target screen image and the second target screen image, and change detection is performed on the grid contents of the non-occluded regions to obtain a screen change result. Through change detection on the grid contents of the non-occluded regions, when the number of grids with image change exceeds a threshold value, it is considered that the screen has changed, thereby improving the accuracy of screen change detection under the condition of occlusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of screen detection, in particular to a screen change detection method and device in a shielding case. BACKGROUND

[0002] With the progress of society, the business of banks has become diverse, not just simple businesses such as cash deposit and withdrawal, but also the introduction of security verification measures such as password modification, face recognition, identity card information reading, and complex businesses such as financial products. As a result, super teller machines have emerged, which can meet most of the needs of customers and only need to operate the super teller machine to carry out bank business, greatly improving work efficiency. In the existing bank super teller machine operation scene, since it is a violation of regulations for bank staff to operate the super teller machine on behalf of the customer, detection of such customer representation is needed to avoid problems such as account security caused by improper process operation.

[0003] In the prior art, whether there is a violation of customer representation is often determined by using the screen change of the super teller machine in the two consecutive frames of images in the monitoring video. However, when there is a case of partial shielding of the super teller machine screen, the existing technical means cannot accurately detect the screen change, resulting in a decrease in the accuracy of screen change detection. SUMMARY

[0004] Therefore, the present application provides a screen change detection method and device in a shielding case, which aims to improve the accuracy of screen change detection in a shielding case.

[0005] To solve the above problems, the technical scheme provided by the present application is as follows:

[0006] A screen change detection method in a shielding case, the method comprising:

[0007] obtaining a first image and a second image, wherein the first image and the second image are two consecutive images;

[0008] cutting a screen region from the first image and the second image and performing perspective transformation to obtain a first target screen image and a second target screen image;

[0009] using a segmentation algorithm to identify the shielded region and the unshielded region in the first target screen image and the second target screen image;

[0010] adding a plurality of grids in the first target screen image and the second target screen image, respectively;

[0011] detecting changes in the grids of the unshielded region in the first target screen image and the second target screen image;

[0012] determining whether the number or proportion of changed grids exceeds a first threshold value, and if yes, determining that the screen has changed, and if no, determining that the screen has not changed.

[0013] In a possible implementation, the obtaining of the screen regions from the first image and the second image and the perspective transformation to obtain the first target screen image and the second target screen image include:

[0014] obtaining the screen positions from the first image and the second image by using a detection algorithm or manual labeling;

[0015] transforming the screen regions corresponding to the screen positions into the first target screen image and the second target screen image by using a perspective transformation method, so that the first target screen image and the second target screen image are transversely horizontal and longitudinally vertical.

[0016] In a possible implementation, the identifying of the occluded regions and the non-occluded regions of the first target screen image and the second target screen image by using a segmentation algorithm includes:

[0017] identifying the occluded regions and the non-occluded regions of the first target screen image and the second target screen image by using a model based on deep learning or a segmentation algorithm based on a traditional method.

[0018] In a possible implementation, the identifying of the occluded regions and the non-occluded regions of the first target screen image and the second target screen image by using a segmentation algorithm includes:

[0019] identifying the occluded regions of the first target screen image and the second target screen image by using a segmentation algorithm;

[0020] taking a union set of the occluded regions of the first target screen image and the second target screen image;

[0021] filling the occluded regions in the union set back into the first target screen image and the second target screen image to obtain the non-occluded regions.

[0022] In a possible implementation, before the identifying of the occluded regions and the non-occluded regions of the first target screen image and the second target screen image by using a segmentation algorithm, the method further includes:

[0023] performing morphological denoising on the first target screen image and the second target screen image.

[0024] In a possible implementation, the change detection on the grids of the non-occluded regions in the first target screen image and the second target screen image includes:

[0025] determine whether the pixels contained in the same position grid of the un-occluded area in the first target screen image and the second target screen image change;

[0026] If the number or proportion of pixels in the grid changes exceeds a second threshold value, it is considered that the grid has changed, otherwise it is considered that the grid has not changed.

[0027] An apparatus for detecting screen change in an occlusion case, the apparatus comprising:

[0028] an image acquisition module configured to acquire a first image and a second image, wherein the first image and the second image are two consecutive images;

[0029] a screen area processing module configured to crop a screen area from the first image and the second image and perform perspective transformation to obtain a first target screen image and a second target screen image;

[0030] an image segmentation module configured to identify an occluded area and an un-occluded area in the first target screen image and the second target screen image using a segmentation algorithm;

[0031] a grid adding module configured to add a plurality of grids in the first target screen image and the second target screen image, respectively;

[0032] a grid change detection module configured to perform change detection on the grids of the un-occluded area in the first target screen image and the second target screen image;

[0033] a determination module configured to determine whether the number or proportion of changed grids exceeds a first threshold value, and if yes, determine that the screen has changed, otherwise determine that the screen has not changed.

[0034] In a possible implementation, the image segmentation module comprises:

[0035] an identification unit configured to identify the occluded area in the first target screen image and the second target screen image using a segmentation algorithm;

[0036] an occluded area operation unit configured to take the union of the occluded area in the first target screen image and the second target screen image;

[0037] an un-occluded area determination unit configured to backfill the occluded area after the union into the first target screen image and the second target screen image to obtain an un-occluded area.

[0038] An apparatus for detecting screen change in an occlusion case, the apparatus comprising:

[0039] a memory configured to store instructions or codes of the apparatus for detecting screen change in an occlusion case.

[0040] a processor configured to execute instructions or codes for detecting the screen change in the occlusion case to implement the above-mentioned method for detecting the screen change in the occlusion case.

[0041] A computer storage medium, in which codes are stored, when the codes are executed, a device executing the codes implements the above-mentioned method for detecting the screen change in the occlusion case.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] The present application first acquires a first image and a second image in a task scenario, wherein the first image and the second image refer to two consecutive images; then acquires a screen region in the first image and the second image and performs perspective transformation to obtain a first target screen image and a second target screen image; then uses a segmentation algorithm to segment the occluded region and the non-occluded region of the first target screen image and the second target screen image; finally adds at least one grid in the first target screen image and the second target screen image; and detects the grid content of the non-occluded region for change to obtain a screen change result.

[0044] The present application detects the grid content of the non-occluded region for change, and when the number of images changed exceeds a threshold value, it is considered that the screen has changed, thereby improving the accuracy of screen change detection in the occlusion case. BRIEF DESCRIPTION OF DRAWINGS

[0045] To make the technical solutions in the present embodiment or prior art clearer, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0046] Figure 1 A method flowchart of the method for detecting the screen change in the occlusion case provided by the present embodiment;

[0047] Figure 2 A method flowchart of the method for detecting the screen change in the occlusion case in a specific application scenario provided by the present embodiment;

[0048] Figure 3 A schematic diagram of two consecutive images in a task scenario provided by the present embodiment;

[0049] Figure 4 A schematic diagram of the screen region in the acquired first image and second image provided by the present embodiment;

[0050] Figure 5 A schematic diagram of a first target screen image and a second target screen image provided by an embodiment of the present application;

[0051] Figure 6 A schematic diagram of an occluded area and an unoccluded area identified by a meanshift method provided by an embodiment of the present application;

[0052] Figure 7 A schematic diagram of a result after morphological denoising provided by an embodiment of the present application;

[0053] Figure 8 A schematic diagram of a result after taking a union of the occluded areas provided by an embodiment of the present application;

[0054] Figure 9 A schematic diagram of filling the occluded areas after taking a union back into the target screen image provided by an embodiment of the present application;

[0055] Figure 10 A schematic diagram of adding a grid to the target screen image provided by an embodiment of the present application;

[0056] Figure 11 A schematic diagram of a structure of a detection device for screen changes in a case of occlusion provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] In the prior art, whether a customer operation occurs is usually confirmed by using screen change conditions of a super teller machine in two continuous frames of images in a monitoring video, for example, two frames of images are selected in the video for comparison, and whether a customer operation occurs is confirmed by directly observing whether the content of the super teller machine screen changes.

[0058] After research, compared with an actual screen, a video picture has relatively large background interference, and at this time, when the super teller machine screen is partially occluded, the existing technical means cannot make accurate screen change detection, resulting in reduced accuracy of screen change detection.

[0059] The present application first acquires a first image and a second image in a task scene, wherein the first image and the second image refer to two continuous frames of images; then acquires a screen area in the first image and the second image and performs perspective transformation to obtain a first target screen image and a second target screen image; then divides an occluded area and an unoccluded area of the first target screen image and the second target screen image by using a segmentation algorithm; finally, adds at least one grid to the first target screen image and the second target screen image; and detects changes in the grid content of the unoccluded area to obtain a screen change result.

[0060] The application detects the grid content in the un-occluded area, and considers that the screen changes when the number of grids with image changes exceeds a threshold, thereby improving the accuracy of screen change detection in the occluded case.

[0061] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the application.

[0062] Referring to Figure 1 , Figure 1 A method flowchart of the method for detecting screen changes in the occluded case provided by the embodiments of the application includes the following steps.

[0063] S101: Obtain a first image and a second image.

[0064] The first image and the second image are two consecutive images, for example, two consecutive images obtained by cutting a video.

[0065] The obtained first image and second image serve as the basis for subsequent image recognition.

[0066] S102: Cut the screen area from the first image and the second image and perform perspective transformation to obtain a first target screen image and a second target screen image.

[0067] The perspective transformation specifically refers to a method for changing the screen area image in the image into a horizontal and vertical rectangular image, including rotation, translation, projection transformation and other specific means.

[0068] In a possible implementation, cutting the screen area from the first image and the second image and performing perspective transformation to obtain the first target screen image and the second target screen image includes the following steps.

[0069] The screen position is obtained by using a detection algorithm or a manual labeling method.

[0070] The screen area in the screen position is transformed into the horizontal and vertical first target screen image and the second target screen image by using the perspective transformation method.

[0071] The detection algorithm refers to intelligent identification of the screen area by using an existing algorithm, and the manual labeling method refers to manual drawing of the screen position.

[0072] S103: Identify the occluded area and the un-occluded area in the first target screen image and the second target screen image by using a segmentation algorithm.

[0073] Wherein, the segmentation algorithm refers to an algorithm for separating images by technical means.

[0074] In a possible implementation, the step of segmenting the occluded area and the non-occluded area of the first target screen image and the second target screen image by using the segmentation algorithm comprises:

[0075] The occluded area and the non-occluded area of the first target screen image and the second target screen image are segmented by using a model based on deep learning or a segmentation algorithm based on a traditional method.

[0076] Wherein, there are many segmentation algorithms in the prior art, including models based on deep learning (Mask-RCNN / FCN / DeepLabv3+ etc.) and segmentation based on traditional methods (threshold-based segmentation method, watershed algorithm, etc.). The advantage of the model based on deep learning is good segmentation effect, and the disadvantage is that it occupies many resources and has low computing efficiency, which is not conducive to deployment. The advantage of the segmentation based on traditional methods is that it occupies fewer resources and is convenient for edge device deployment, and the disadvantage is that the segmentation effect is general and seed points need to be selected in advance; since there are many segmentation algorithms, the specific segmentation algorithm used is not limited in the embodiments of the present application.

[0077] In a possible implementation, the step of identifying the occluded area and the non-occluded area of the first target screen image and the second target screen image by using the segmentation algorithm further comprises:

[0078] The first target screen image and the second target screen image are morphologically denoised.

[0079] Wherein, morphological denoising refers to reducing background noise in a digital image. In simple terms, it reduces the useless background interference information in the identified image.

[0080] In a possible implementation, the step of identifying the occluded area and the non-occluded area of the first target screen image and the second target screen image by using the segmentation algorithm comprises:

[0081] The occluded area of the first target screen image and the second target screen image is identified by using the segmentation algorithm.

[0082] The occluded area of the first target screen image and the second target screen image is taken as a union set.

[0083] The occluded area after the union set is backfilled into the first target screen image and the second target screen image to obtain the non-occluded area.

[0084] The purpose of taking the union of the occluded areas of the first target screen image and the second target screen image is to determine the size of the occluded areas in the two images, and the determined occluded areas after the union will not participate in subsequent grid change detection, thereby improving the detection accuracy. If this step is not taken, when the occluded areas of the first target screen image and the second target screen image are inconsistent, the inconsistent positions of the occluded areas will also be determined as having changed, and the accuracy of the detection result will be lowered.

[0085] The purpose of backfilling the occluded areas after the union to the first target screen image and the second target screen image is similar to the control variable method, that is, only the areas in the two images that are not occluded are used for grid change detection, which is more rigorous.

[0086] S104: Add a plurality of grids in the first target screen image and the second target screen image, respectively.

[0087] Adding a plurality of grids means adding at least one grid, and this step is a preliminary preparation for subsequent grid change detection.

[0088] S105: Perform change detection on the grids in the unoccluded areas of the first target screen image and the second target screen image.

[0089] The change detection refers to judging whether the image content in the grid changes.

[0090] In one possible implementation, the change detection on the grids in the unoccluded areas of the first target screen image and the second target screen image includes:

[0091] Judging whether the pixels contained in the grids in the same positions in the unoccluded areas of the first target screen image and the second target screen image change.

[0092] If the number of changed pixels in the grid exceeds the second threshold, it is considered that the grid has changed, otherwise it is considered that the grid has not changed.

[0093] Each grid contains at least one pixel. When the content of a single pixel changes, it is considered that the pixel has changed. When the number or proportion of changed pixels in a single grid exceeds the second threshold, it is considered that the content of the grid has changed. For example, if more than 50% of the pixels in the grid change, it is considered that the grid has changed.

[0094] S106: Judging whether the number of changed grids exceeds the first threshold. If yes, it is determined that the screen has changed, otherwise it is determined that the screen has not changed.

[0095] The step is to continue judging the final screen change according to the number of changed grids obtained in step S105, wherein the first threshold value can be set as the number of changed grids or the proportion of the number of changed grids in the total number of grids in the non-occluded area.

[0096] The embodiment of the present application detects the change of the grid content in the non-occluded area, and considers that the screen changes when the number of grids with image changes exceeds the first threshold value, thereby improving the accuracy of screen change detection in the occluded case.

[0097] The above describes the method for detecting screen change in the occluded case provided by the embodiment of the present application, and the method is exemplarily described in combination with a specific application scenario.

[0098] Referring to Figure 2 , Figure 2 The flowchart of the method for detecting screen change in the specific application scenario provided by the embodiment of the present application, assuming that the specific use scenario of the embodiment of the present application is a bank scenario, specifically including the following steps:

[0099] S201: Obtain a first image and a second image.

[0100] The first image and the second image refer to the images in two consecutive task scenarios as shown in Figure 3 Figure 3 The left side of FIG. 2 is the first image, and the right side is the second image. It can be seen that the screen area in the first image is occluded by the customer's hand.

[0101] S202: Obtain the screen area in the first image and the second image, and perform perspective transformation on the obtained screen area to obtain a first target screen image and a second target screen image.

[0102] Referring to Figure 4 , Figure 4 The screen area in the obtained first image and the second image, Figure 5 The first target screen image and the second target screen image obtained after perspective transformation.

[0103] S203: Use the meanshift method to identify the occluded area and the non-occluded area in the first target screen image and the second target screen image and perform morphological denoising.

[0104] The meanshift method is relatively mature, and in the embodiment of the present application, the idea of classification is mainly used, that is, the occluded area is one class, and the other class is the non-occluded area. The screen area is classified by judging the screen brightness and other indicators to obtain the black area (occluded area) and white area (non-occluded area) as shown in Figure 6 ​​

[0105] Of course, other existing methods can also be used for the identification of the occluded area, and the identification method of the occluded area is not specifically limited in the embodiments of the present application.

[0106] The morphological denoising result is shown in FIG. 6B. As this step only needs to determine the occluded area, the denoising mainly removes the interference of the screen content. Figure 7

[0107] S204: Taking the union of the occluded areas and backfilling to the first target screen image and the second target screen image.

[0108] The result of taking the union is shown in FIG. 6C. As can be seen from FIG. 6C, Figure 8 the black area in FIG. 6C is obtained by taking the union of the denoising results of FIG. 6B. Figure 8 Figure 7

[0109] The schematic diagram of backfilling the occluded area after taking the union to the first target screen image and the second target screen image is shown in FIG. 6D. Figure 9 As can be seen from FIG. 6D, Figure 9 the black area in FIG. 6D is obtained by filling the black area in FIG. 6C to the target screen image. Figure 8

[0110] S205: Adding a grid to the first target screen image and the second target screen image and performing grid change detection.

[0111] The schematic diagram of adding a grid is shown in FIG. 6E. The grid change detection is only performed on the non-occluded area in the figure. Figure 10

[0112] S206: Determining whether the number or proportion of changed grids exceeds a preset threshold, and if yes, determining that the image has changed, otherwise, determining that the image has not changed.

[0113] The embodiments of the present application combine specific application scenarios to specifically describe the detection method of screen change under the occlusion condition provided by the present application. It can be seen from the embodiments that the detection method of screen change under the occlusion condition provided by the present application can be applied to life scenarios and has high generalization.

[0114] The above are some specific implementation manners of the detection method of screen change under the occlusion condition provided by the embodiments of the present application. Based on this, the present application also provides a corresponding device. The device provided by the embodiments of the present application will be introduced from the perspective of functional modularization.

[0115] Referring to the structural schematic diagram of the detection device of screen change under the occlusion condition shown in FIG. 7, the device comprises: Figure 11

[0116] ​​​​​​The image acquisition module 1101 is configured to acquire a first image and a second image; wherein the first image and the second image are two frames of continuous images.

[0117] The screen region processing module 1102 is configured to crop a screen region from the first image and the second image and perform perspective transformation to obtain a first target screen image and a second target screen image.

[0118] The image segmentation module 1103 is configured to identify an occluded region and an unoccluded region in the first target screen image and the second target screen image by using a segmentation algorithm.

[0119] The grid adding module 1104 is configured to add a plurality of grids in the first target screen image and the second target screen image, respectively.

[0120] The grid change detection module 1105 is configured to perform change detection on the grids in the unoccluded region in the first target screen image and the second target screen image.

[0121] The judgment module 1106 is configured to judge whether the number or proportion of changed grids exceeds a first threshold value, and if yes, determine that the screen has changed, and if not, determine that the screen has not changed.

[0122] In a possible implementation, the screen region processing module 1102 comprises:

[0123] A screen position detection unit is configured to acquire a screen position by using a detection algorithm or a manual labeling method.

[0124] A screen transformation unit is configured to transform a screen region in the screen position into a first target screen image and a second target screen image with a horizontal direction being horizontal and a vertical direction being vertical by using a perspective transformation method.

[0125] In a possible implementation, the image segmentation module 1103 is specifically configured to:

[0126] Identify the occluded region and the unoccluded region in the first target screen image and the second target screen image by using a model based on deep learning or a segmentation algorithm based on a traditional method.

[0127] In a possible implementation, the image segmentation module 1103 comprises:

[0128] An identification unit is configured to identify the occluded region in the first target screen image and the second target screen image by using a segmentation algorithm.

[0129] An occluded region operation unit is configured to take a union of the occluded region in the first target screen image and the second target screen image.

[0130] The unoccluded area determination unit is configured to backfill the occluded area after the set operation into the first target screen image and the second target screen image to obtain an unoccluded area.

[0131] In a possible implementation, the image segmentation module 1103 further includes:

[0132] The morphological denoising module is configured to perform morphological denoising on the first target screen image and the second target screen image.

[0133] In a possible implementation, the grid change detection module 1105 includes:

[0134] The determination unit is configured to determine whether the pixels contained in the same position grid of the unoccluded area in the first target screen image and the second target screen image change.

[0135] The grid change result acquisition unit is configured to determine the grid change condition according to the determination result, and consider that the grid changes if the number or proportion of the pixels in the grid changes exceeds a second threshold value, or consider that the grid does not change.

[0136] The embodiments of the present application improve the accuracy of screen change detection in the case of occlusion by detecting the change of the grid content of the unoccluded area, and considering that the screen changes when the number of grids with image changes exceeds a threshold value.

[0137] The embodiments of the present application also provide a corresponding device and a computer storage medium for implementing the schemes provided by the embodiments of the present application.

[0138] The device includes a memory and a processor, the memory is configured to store instructions or codes, and the processor is configured to execute the instructions or codes to enable the device to perform the method for detecting screen change in the case of occlusion according to any of the embodiments of the present application.

[0139] The computer storage medium stores codes, and when the codes are executed, the device running the codes implements the method for detecting screen change in the case of occlusion according to any of the embodiments of the present application.

[0140] The terms "first" and "second" in the names mentioned in the embodiments of the present application are only used for name identification, and do not represent the first and second in the order.

[0141] Those skilled in the art can clearly understand the whole or part of the steps in the above-mentioned embodiment methods by means of software and a general hardware platform based on the description of the above embodiments. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the method described in each embodiment or some parts of the embodiments of the present application.

[0142] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the description of the method embodiments. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0143] The above-mentioned is only an exemplary embodiment of the present application, and is not used to limit the protection scope of the present application.

Claims

1. A method for detecting screen changes under occlusion, characterized in that, The method includes: Acquire a first image and a second image, wherein the first image and the second image are two consecutive frames; Screen regions are cropped from the first image and the second image and perspective transformation is performed to obtain a first target screen image and a second target screen image; The segmentation algorithm is used to identify the occluded and unoccluded regions in the first target screen image and the second target screen image; Multiple grids are added to the first target screen image and the second target screen image respectively; Change detection is performed on the grid in the unoccluded areas of the first and second target screen images; Determine whether the number or proportion of changing grids exceeds the first threshold. If it does, determine that the screen has changed; otherwise, determine that the screen has not changed. The step of using a segmentation algorithm to identify occluded and unoccluded regions in the first target screen image and the second target screen image includes: The occluded areas in the first target screen image and the second target screen image are identified using a segmentation algorithm; Take the union of the occluded areas of the first target screen image and the second target screen image; The occluded area after taking the union is backfilled into the first target screen image and the second target screen image to obtain the unoccluded area.

2. The method according to claim 1, characterized in that, The step of obtaining the first target screen image and the second target screen image by acquiring the screen area from the first image and the second image and performing perspective transformation includes: The screen position is obtained from the first image and the second image using detection algorithms or manual annotation methods; The screen area corresponding to the screen position is transformed into a first target screen image and a second target screen image, which are horizontally horizontal and vertically vertical, using the perspective transformation method.

3. The method according to claim 1, characterized in that, The step of using a segmentation algorithm to identify the occluded and unoccluded regions of the first and second target screen images includes: The occluded and unoccluded regions of the first and second target screen images are identified using deep learning-based models or segmentation algorithms based on traditional methods.

4. The method according to claim 1, characterized in that, Before using the segmentation algorithm to identify the occluded and unoccluded regions of the first and second target screen images, the process further includes: Morphological denoising is performed on the first target screen image and the second target screen image.

5. The method according to claim 1, characterized in that, The change detection of the grid in the unoccluded area of ​​the first target screen image and the second target screen image includes: Determine whether the pixels contained in the same position grid of the unoccluded area in the first target screen image and the second target screen image have changed; If the number or proportion of changes in pixels within a grid exceeds the second threshold, the grid is considered to have changed; otherwise, the grid is considered not to have changed.

6. A device for detecting screen changes under obstructed conditions, characterized in that, The device includes: The image acquisition module is used to acquire a first image and a second image; wherein the first image and the second image are two consecutive frames; The screen area processing module is used to extract screen areas from the first image and the second image and perform perspective transformation to obtain a first target screen image and a second target screen image. An image segmentation module is used to identify occluded and unoccluded regions in the first target screen image and the second target screen image using a segmentation algorithm. A grid adding module is used to add multiple grids to the first target screen image and the second target screen image respectively; The grid change detection module is used to detect changes in the grid in the unoccluded areas of the first target screen image and the second target screen image. The judgment module is used to determine whether the number or proportion of the changing grids exceeds the first threshold. If it does, the screen is determined to have changed; otherwise, the screen is determined not to have changed. The image segmentation module includes: The recognition unit is used to identify the occluded areas in the first target screen image and the second target screen image using a segmentation algorithm. An occlusion region operation unit is used to combine the occluded regions of the first target screen image and the second target screen image. The unobstructed area determination unit is used to backfill the obstructed area after taking the union into the first target screen image and the second target screen image to obtain the unobstructed area.

7. A device for detecting screen changes under obstructed conditions, characterized in that, The device includes: A memory for storing instructions or code for detecting screen changes under occlusion conditions; A processor is configured to execute instructions or code for detecting screen changes under occlusion conditions, in order to implement the screen change detection method under occlusion conditions as described in any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer storage medium stores code, and when the code is executed, the device running the code implements the screen change detection method under occlusion conditions as described in any one of claims 1-5.

Images