Picture tracking shielding region division method and device, storage medium and equipment

Abstract

Embodiments of the present application provide a picture tracking shielding area division method, device, storage medium and equipment, the method comprises: constructing a plurality of groups of classroom real three-dimensional graphs corresponding to a classroom camera device; image matching is carried out in the classroom three-dimensional reconstruction material library, and the target classroom three-dimensional reconstruction material graph corresponding to each group of classroom real three-dimensional graphs is obtained; according to each group of target classroom three-dimensional reconstruction material graphs, a plurality of groups of first picture tracking shielding areas are obtained; each group of classroom real three-dimensional graphs is identified respectively, and a plurality of groups of second picture tracking shielding areas are obtained; according to each group of second picture tracking shielding areas, the first picture tracking shielding area of the corresponding group is adjusted, and a plurality of groups of third picture tracking shielding areas are obtained; according to each group of third picture tracking shielding areas and the first picture tracking shielding area of the corresponding group, the target tracking shielding area of the classroom camera device is determined, and the strong dependence on the camera device can be reduced, and the deployment efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of image recognition, and in particular to a method, apparatus, storage medium, and device for dividing an image tracking shielding area. Background Technology

[0002] Classroom recording equipment has become an essential tool for evaluating teachers' teaching levels and reinforcing students' learning. To monitor and record teaching activities, several cameras are typically used for filming and tracking. During the installation and deployment phase of classroom recording equipment, the deployment personnel will appropriately set and adjust the tracking and shielding areas of the camera equipment based on the size of the classroom, the placement / method of the podium, or whether it is a confidential / private space, in order to better facilitate recording.

[0003] In the process of developing this invention, the inventors discovered that traditional technologies suffer from at least the following problems: Installing and deploying classroom recording equipment requires manual debugging by professional personnel, resulting in high installation and deployment costs, being time-consuming, labor-intensive, and inefficient. Furthermore, prior training and practice for these personnel is necessary, leading to high operational costs for businesses. Recording equipment needs to be compatible with camera equipment from multiple manufacturers, which indirectly requires deployment personnel to use the specific debugging tools for each camera. This creates a strong dependence on the camera equipment, is difficult to learn, and even presents challenges due to differing debugging methods and steps between various tools, resulting in lengthy, cumbersome, and inefficient deployments each time. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this application provides a method, apparatus, storage and device for dividing the image tracking shielding area. It does not require professional deployment personnel for manual debugging, can conveniently and quickly determine the tracking shielding area, avoid a large amount of repetitive and tedious deployment and debugging work, reduce the strong dependence on camera equipment, and improve deployment efficiency.

[0005] According to a first aspect of the embodiments of this application, a method for dividing a screen tracking shielding region is provided, comprising the following steps:

[0006] Construct several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment; the several sets of real-world 3D images of the classroom are determined based on classroom images captured by the classroom camera equipment from several shooting angles;

[0007] Image matching is performed in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of the classrooms in each group; based on the target classroom 3D reconstruction material images in each group, several corresponding first-screen tracking and shielding areas are obtained;

[0008] Each group of classroom reality 3D images is tracked and shielded object identification to obtain several corresponding second screen tracking and shielding areas;

[0009] Based on the second screen tracking shielding area of ​​each group, adjust the first screen tracking shielding area of ​​the corresponding group to obtain several groups of third screen tracking shielding areas.

[0010] The target tracking shielding area of ​​the classroom camera device is determined based on the third screen tracking shielding area of ​​each group and the first screen tracking shielding area of ​​the corresponding group.

[0011] According to a second aspect of the embodiments of this application, a screen tracking shielding region segmentation device is provided, comprising:

[0012] The real-world 3D image construction module is used to construct several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment; the several sets of real-world 3D images of the classroom are determined based on classroom images captured by the classroom camera equipment from several shooting angles;

[0013] The first region determination module is used to perform image matching in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms; and to obtain several corresponding first screen tracking and shielding regions based on the target classroom 3D reconstruction material images of each group.

[0014] The second region determination module is used to identify the tracking and shielding objects in each group of the classroom reality 3D map to obtain several corresponding second screen tracking and shielding regions.

[0015] The third region determination module is used to adjust the first screen tracking shielding area of ​​the corresponding group according to the second screen tracking shielding area of ​​each group, so as to obtain several groups of third screen tracking shielding areas.

[0016] The target area determination module is used to determine the target tracking shielding area of ​​the classroom camera device based on the third screen tracking shielding area of ​​each group and the first screen tracking shielding area of ​​the corresponding group.

[0017] According to a third aspect of the embodiments of this application, an electronic device is provided, including a processor and a memory; the memory stores a computer program adapted to be loaded by the processor and executed as described above in the screen tracking masking region division method.

[0018] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, it implements the screen tracking masking region division method as described above.

[0019] This application embodiment constructs several sets of real-world 3D classroom images corresponding to classroom camera equipment. These sets of images are determined based on classroom images captured by the classroom camera equipment from several shooting angles. Image matching is performed in a classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match each set of real-world 3D classroom images. Based on each set of target classroom 3D reconstruction material images, several sets of first-view tracking shielding areas are obtained. Tracking shielding object identification is performed on each set of real-world 3D classroom images to obtain several sets of second-view tracking shielding areas. Based on each set of second-view tracking shielding areas, the corresponding first-view tracking shielding areas are adjusted to obtain several sets of third-view tracking shielding areas. Based on each set of third-view tracking shielding areas and the corresponding first-view tracking shielding areas, the target tracking shielding area of ​​the classroom camera equipment is determined. This eliminates the need for manual debugging by professional deployment personnel, allowing for convenient and quick determination of the tracking shielding area. It avoids a large amount of repetitive and tedious deployment and debugging work, reduces strong dependence on camera equipment, and improves deployment efficiency.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.

[0021] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating an application scenario of the image tracking shielding region division method according to an embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating a method for dividing a screen tracking shielding area according to an embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating a method for constructing a realistic 3D model of a classroom, as shown in one embodiment of this application;

[0026] Figure 4 is a schematic diagram of the first screen tracking shielding area shown in an embodiment of this application;

[0027] Figure 5 This is a flowchart illustrating a method for determining a target tracking shielding area according to one embodiment of this application;

[0028] Figure 6 This is a schematic block diagram illustrating a screen tracking shielding area division device according to one embodiment of this application;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device according to one embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Wherein, when the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0031] It should be understood that the embodiments described below do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0032] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, in the description of this application, unless otherwise stated, “a plurality” means two or more. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B, which can represent: A alone, A and B together, and B alone; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0033] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms, and these terms are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Depending on the context, the word "if" as used in this application can be interpreted as "when," "when," or "in response to determination."

[0034] Please see Figure 1This is a schematic diagram illustrating an application scenario of the image tracking shielding area division method according to an embodiment of this application. The application scenario includes the image tracking shielding area division method provided in this embodiment, comprising an operation and maintenance client 10, a recording and broadcasting device 20, a classroom camera device 30, and an operation and maintenance server 40.

[0035] The maintenance client 10 and the recording equipment 20 can establish a data communication link via wired or wireless means to enable communication data interaction between the maintenance client 10 and the recording equipment 20. The classroom camera equipment 30 can also establish a data communication link with the recording equipment 20 via wired or wireless means to allow the recording equipment 20 to acquire classroom images captured by the classroom camera equipment 30. The maintenance server 40 can also establish a data communication link with the recording equipment 20 to send the 3D reconstructed classroom footage to the recording equipment 20.

[0036] It should be noted that the classroom camera device 30 in this application embodiment may include a teacher camera device for capturing a full view of the teacher and a student camera device for capturing a full view of the students, etc.

[0037] Understandably, the required image tracking shielding area differs for different classroom camera devices 30. For example, for teacher camera devices, the tracking area typically includes the podium area, while the shielding area usually includes the projector area; for student camera devices, the tracking area typically includes the desk area, while the shielding area usually includes the blackboard area on the back wall of the classroom. However, the principle behind setting the image tracking shielding area is the same for different classroom camera devices 30.

[0038] It should be noted that the operation and maintenance client 10 in this embodiment can be understood as an application installed on a computer device, or it can be understood as a hardware device corresponding to the recording and broadcasting device 20. More specifically, it refers to a computer device, such as a smartphone, a smart interactive whiteboard, or a personal computer. When the client is a mobile device such as a smartphone or a smart interactive whiteboard, the user can install a matching mobile application on the client, or access a web application on the client. When the client is a non-mobile device such as a personal computer (PC), the user can install a matching PC application on the client, or access a web application on the client. This application is used to execute the screen tracking masking area division method of this application. The application can also simply receive screen tracking masking area division instructions sent by the deployment personnel, and then send the screen tracking masking area division instructions to the recording and broadcasting device 20, so that the recording and broadcasting device 20 can execute the screen tracking masking area method of this application.

[0039] The following will be combined with the appendix Figures 2 to 5This paper provides a detailed description of the image tracking shielding area division method provided in the embodiments of this application.

[0040] Please see Figure 2 The image tracking shielding region division method provided in this application includes the following steps:

[0041] Step S101: Construct several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment; the several sets of real-world 3D images of the classroom are determined based on classroom images captured by the classroom camera equipment from several shooting angles.

[0042] It should be understood that classroom camera equipment refers to various types of cameras, including those used to capture a full view of the teacher, close-up shots of the teacher, student cameras to capture a full view of the students, and close-up shots of the students. The required image tracking shielding area varies depending on the type of classroom camera equipment.

[0043] In one optional embodiment, the classroom images may be images captured from several angles by classroom cameras manually controlled by the deployment personnel. In another optional embodiment, the classroom images may also be images captured from several preset angles by the maintenance client controlling the classroom cameras through recording equipment or by the maintenance client directly controlling the teacher's camera.

[0044] Among these, the classroom images captured from several angles can be images taken at preset intervals on both sides of the target tracking area, with the classroom camera directly facing the target tracking area as the center line. Specifically, taking a teacher's camera as an example, if the target tracking area of ​​the teacher's camera is the podium area, then the classroom image captured when the teacher's camera is directly facing the podium, as well as classroom images captured at 30-degree and 60-degree intervals on the left and right sides of the teacher's camera relative to the podium, can be obtained.

[0045] Step S102: Perform image matching in the classroom 3D reconstruction material library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms; based on each group of target classroom 3D reconstruction material images, obtain several corresponding first-screen tracking and shielding areas.

[0046] The classroom 3D reconstruction material library includes 3D reconstruction images of various classroom types in K12 (kindergarten through twelfth grade, also known as basic education).

[0047] In an optional embodiment, the classroom 3D reconstruction material library can be stored in the operation and maintenance client, and then the 3D images of each group of classrooms can be directly matched with the teacher 3D reconstruction material library to obtain the target classroom 3D reconstruction material image that matches the actual 3D images of each group of classrooms, and then the target classroom 3D reconstruction material image is sent to the recording and broadcasting equipment.

[0048] In another optional embodiment, the classroom 3D reconstruction material library can also be stored in the recording and broadcasting device connected to the operation and maintenance client. The recording and broadcasting device directly matches each group of classroom 3D images with the teacher 3D reconstruction material library to obtain the target classroom 3D reconstruction material image that matches the actual 3D images of each group of classrooms.

[0049] In another optional embodiment, the classroom 3D reconstruction material library can also be stored in the operation and maintenance server. The operation and maintenance client sends a matching command to the operation and maintenance server, so that the operation and maintenance server matches the classroom 3D image with the teacher 3D reconstruction material library to obtain the target classroom 3D reconstruction material image that matches the actual 3D image of each group of classrooms. Then, based on the target classroom 3D reconstruction material images, several sets of first-screen tracking shielding areas are obtained. The operation and maintenance server then feeds back the several sets of first-screen tracking shielding areas to the recording and broadcasting equipment.

[0050] In an optional embodiment, the recording device can perform similarity calculations between the feature point information extracted from each set of real-world 3D images and the feature point information extracted from each teacher's 3D reconstruction material image in the classroom 3D reconstruction material image library, thereby obtaining several sets of classroom 3D reconstruction material images that match several sets of real-world 3D images.

[0051] In another optional embodiment, the recording device can also directly calculate the similarity between the pixel information of each set of real 3D images and the 3D reconstruction material images of each teacher in the classroom 3D reconstruction material image library, thereby matching several sets of classroom real 3D images with several sets of classroom 3D reconstruction material images.

[0052] Step S103: Track and identify the objects in each group of classroom reality 3D images to obtain several corresponding second screen tracking and shielding areas.

[0053] It should be understood that different classroom camera devices require different tracking shielding objects. For teacher-centric cameras, the shielding object could be a blackboard or projector, where the teacher may be lecturing via the blackboard or projector. In this case, it's more important to track and record the content on the blackboard or projector, rather than the teacher themselves. Alternatively, the tracking object could be a lectern, where the teacher is lecturing, and in this case, it's more important to track and record the teacher.

[0054] In an optional embodiment, the recording device has several preset shielding object recognition models and tracking object recognition models; several sets of classroom reality 3D images are input into each shielding object recognition model and tracking object recognition model respectively, to determine the position of each shielding object and the position of the tracking object in each set of classroom reality 3D images, thereby obtaining several sets of second screen tracking shielding areas.

[0055] In another optional embodiment, the recording device can also directly determine the position of each shielded object and the position of the tracked object in the real three-dimensional map of the classroom based on the tracking shielded object recognition algorithm, thereby obtaining several sets of second-screen tracking shielded areas.

[0056] Step S104: Based on the tracking masking area of ​​each group of second screens, adjust the tracking masking area of ​​the corresponding group of first screens to obtain several groups of tracking masking areas of third screens.

[0057] Considering that real classrooms may have their own unique layouts and may not be able to match the most suitable 3D reconstruction image in the classroom 3D reconstruction material library, it is necessary to adjust the tracking shielding area of ​​the first screen according to the tracking shielding area of ​​the second screen to improve accuracy.

[0058] One possible adjustment method is to compare the tracking shielding area of ​​the second screen with the tracking shielding area of ​​the first screen, and obtain the third tracking shielding area by expanding or shrinking the tracking shielding area of ​​the second screen according to the position of each tracking shielding area in the corresponding tracking shielding area of ​​the first screen.

[0059] Step S105: Determine the target tracking shielding area of ​​the classroom camera equipment based on the tracking shielding area of ​​the third screen of each group and the tracking shielding area of ​​the first screen of the corresponding group.

[0060] Each group of third-view tracking shielding areas includes several sub-tracking areas and sub-shielding areas, each corresponding to the same tracking object and shielding object. Therefore, based on the matching degree value between the sub-tracking areas and sub-shielding areas and the corresponding first-view tracking shielding areas, the sub-tracking areas and sub-shielding areas that best match the tracking object and shielding object can be selected from several groups of third-view tracking shielding areas. Then, the best-matching sub-tracking areas and sub-shielding areas are combined to obtain the target tracking shielding area of ​​the classroom camera equipment, thereby further improving the accuracy of tracking shielding area determination.

[0061] This application embodiment constructs several sets of real-world 3D classroom images corresponding to classroom camera equipment. These sets of real-world 3D classroom images are determined based on classroom images captured by the classroom camera equipment from several shooting angles. Image matching is performed in a classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match each set of real-world 3D classroom images. Based on each set of target classroom 3D reconstruction material images, several sets of first-screen tracking shielding areas are obtained. Tracking shielding object identification is performed on each set of real-world 3D classroom images to obtain several sets of second-screen tracking shielding areas. Based on each set of second-screen tracking shielding areas, the corresponding first-screen tracking shielding areas are adjusted to obtain several sets of third-screen tracking shielding areas. Based on each set of third-screen tracking shielding areas and the corresponding first-screen tracking shielding areas, the target tracking shielding area of ​​the classroom camera equipment is determined. This eliminates the need for manual debugging by professional deployment personnel, allowing for convenient and quick determination of the tracking shielding area. It avoids a large amount of repetitive and tedious deployment and debugging work, reduces the strong dependence on camera equipment, and improves deployment efficiency.

[0062] Please see Figure 3 In an optional embodiment, step S101, which involves constructing several sets of real-world 3D classroom images corresponding to the classroom camera equipment, includes:

[0063] Step S1011: Receive classroom environment size information input by the user.

[0064] The classroom environment size information includes one or more of the following: the length, width and height of the classroom, the length and width of the blackboard, and the length and width of the projector.

[0065] The classroom environment dimensions can be obtained by the operations and maintenance client through a connected size measurement device. Alternatively, the deployment personnel can manually input the classroom environment dimensions into the operations and maintenance client. Specifically, the deployment personnel can log in to the operations and maintenance application on the operations and maintenance client, enter the product serial number of the recording and broadcasting equipment to identify the recording and broadcasting equipment for the screen tracking shielding area, and then, in response to the trigger operation of screen tracking shielding area division, receive the classroom environment dimensions input by the user. In other words, the deployment personnel only need to input the classroom environment dimensions to conveniently and quickly obtain the screen tracking shielding area.

[0066] Step S1012: Based on the classroom environment size information and classroom images taken by the classroom camera equipment from several shooting angles, construct several sets of real-world 3D maps of the classroom corresponding to the classroom camera equipment.

[0067] The construction of a realistic 3D model of the classroom, based on the classroom environment size information and classroom images, can be achieved using distorted perspective or other methods that can determine the realistic 3D model of the classroom; this application does not impose any restrictions on this.

[0068] This application embodiment constructs several sets of real-world 3D images of the classroom based on the classroom environment size information input by the user and classroom images captured by the classroom camera equipment from several shooting angles, thereby obtaining the target tracking shielding area. Therefore, there is no need for professional deployment personnel to manually debug; only the classroom environment size information needs to be input to conveniently and quickly determine the tracking shielding area.

[0069] In an optional embodiment, step S102, which involves performing image matching in a classroom 3D reconstruction material image library to obtain a target classroom 3D reconstruction material image that matches the actual 3D images of each group of classrooms, includes:

[0070] Step S1021: Match each group of classroom reality 3D images with each classroom 3D reconstruction material image in the classroom 3D reconstruction material image library, and determine the classroom 3D reconstruction material image with the highest similarity as the target classroom 3D reconstruction material image for matching the corresponding group of classroom reality 3D images.

[0071] This application embodiment calculates the similarity between the actual 3D classroom image and the 3D reconstruction images of various classrooms in the classroom 3D reconstruction image library, which can quickly and accurately determine the target classroom 3D reconstruction image that matches each group of actual 3D classroom images.

[0072] In an optional embodiment, step S102, which involves obtaining several sets of first-screen tracking shielding areas based on the 3D reconstruction material images of each set of target classrooms, includes:

[0073] Step S1022: Based on the mapping relationship between the 3D reconstruction material images of each classroom and the screen tracking shielding area in the classroom 3D reconstruction material image library, obtain the first screen tracking shielding area corresponding to each group of target classroom 3D reconstruction material images.

[0074] As shown in Figures 4(1), 4(2) and 4(3), at least three types of first-screen tracking shielding areas are preset in the operation and maintenance server. Among them, the areas where the letters A, B, C and D are located are shielding areas, and the blank areas are tracking areas.

[0075] This application embodiment can quickly determine the first screen tracking shielding area corresponding to each group of target classroom 3D reconstruction material images by using the mapping relationship between each classroom 3D reconstruction material image in the preset classroom 3D reconstruction material image library and the screen tracking shielding area.

[0076] Please see Figure 5 In an optional embodiment, step S105, which involves determining the target tracking shielding area of ​​the classroom camera device based on the tracking shielding areas of each group of third-view images and the corresponding group of first-view images, includes:

[0077] Step S1051: Determine the matching position of each sub-shielding area based on the matching degree of each sub-shielding area in the third tracking shielding area of ​​each group relative to each sub-shielding area in the first screen tracking shielding area of ​​the corresponding group;

[0078] Step S1052: Determine the target tracking shielding area of ​​the classroom camera equipment based on the matching position of each shielding area.

[0079] Each group of third-view tracking shielding areas includes several sub-shielded areas. The same sub-shielded area in each group corresponds to the same shielded object. The accuracy of determining the shielded object for each sub-shielded area varies within each group. For the first group of third-view tracking areas, the sub-shielded area determined for the first shielded object is more accurate; for the second group, the sub-shielded area determined for the second shielded object is more accurate. Therefore, based on the matching degree value between the sub-shielded area and the corresponding first-view tracking shielding area, the sub-shielded area that best matches each shielded object can be selected from several groups of third-view tracking shielding areas. These best-matching sub-shielded areas are then combined to obtain the target tracking shielding area for the classroom camera equipment, thereby further improving the accuracy of the tracking shielding area determination. It is understandable that the shielding area and the tracking area are relative concepts; once the shielding area is determined, the tracking area can also be determined accordingly. Therefore, in practical implementation, it is sufficient to determine only the shielding area.

[0080] In an optional embodiment, step S1051, which involves determining the matching position of each sub-shielding area based on the matching degree of each sub-shielding area in each group of third tracking shielding areas relative to each sub-shielding area in the corresponding group of first screen tracking shielding areas, includes:

[0081] Step S10511: Calculate the similarity between each sub-mask region in the third tracking mask region of each group and each sub-mask region in the first tracking mask region of the corresponding group. Determine the sub-mask region with the highest similarity in the third tracking mask region of each group as the matching position of the corresponding sub-mask region, and obtain the matching position of each sub-mask region.

[0082] It is understandable that similarity can be calculated using one or more of the following: pixel value similarity, area and shape similarity, and feature information similarity, so as to conveniently determine the location of the sub-shielding region with the highest similarity to each sub-shielding object.

[0083] In an optional embodiment, after step S105, which determines the target tracking shielding area of ​​the classroom camera device based on the tracking shielding areas of each group of third-view images and the tracking shielding areas of the corresponding group of first-view images, the method further includes:

[0084] Step S106: Display the target tracking masking area;

[0085] Step S107: In response to the change operation of each tracking shield position in the target tracking shield area, update the target tracking shield area.

[0086] In this embodiment of the application, after determining the target tracking shielding area, changes to the target tracking shielding area can be received from the user, thereby obtaining a tracking shielding area that is more suitable for the user's needs.

[0087] Please see Figure 6 This is a schematic diagram of the structure of the screen tracking shielding area division device provided in the second embodiment of this application.

[0088] The device 200 includes:

[0089] The real-world 3D image construction module 201 is used to construct several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment; the several sets of real-world 3D images of the classroom are determined based on classroom images captured by the classroom camera equipment from several shooting angles;

[0090] The first region determination module 202 is used to perform image matching in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of the classroom in each group; and to obtain several corresponding first screen tracking and shielding regions based on the target classroom 3D reconstruction material images in each group.

[0091] The second region determination module 203 is used to identify the tracking and shielding objects in each group of the classroom reality three-dimensional map to obtain several corresponding second screen tracking and shielding regions.

[0092] The third region determination module 204 is used to adjust the first screen tracking shielding area of ​​the corresponding group according to the second screen tracking shielding area of ​​each group, so as to obtain several groups of third screen tracking shielding areas.

[0093] The target area determination module 205 is used to determine the target tracking shielding area of ​​the classroom camera device based on the third screen tracking shielding area of ​​each group and the first screen tracking shielding area of ​​the corresponding group.

[0094] The step of performing image matching in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms includes:

[0095] The actual 3D image of each classroom in each group is matched with the 3D reconstruction images of each classroom in the classroom 3D reconstruction material library. The 3D reconstruction image of the classroom with the highest similarity is determined as the target 3D reconstruction image of the corresponding group of classrooms.

[0096] In an optional embodiment, the first region determination module 202 includes a target classroom 3D reconstruction material image acquisition module; the target classroom 3D reconstruction material image acquisition module is used to match each group of the actual 3D images of the classroom with each 3D reconstruction material image of the classroom in the classroom 3D reconstruction material image library, and determine the classroom 3D reconstruction material image with the highest similarity as the target classroom 3D reconstruction material image that matches the actual 3D images of the classroom in the corresponding group.

[0097] In an optional embodiment, the first region determination module 202 further includes a region acquisition module; the region acquisition module is used to obtain the first screen tracking shielding area corresponding to each group of target classroom 3D reconstruction material images based on the mapping relationship between each classroom 3D reconstruction material image and the screen tracking shielding area in the classroom 3D reconstruction material image library.

[0098] In an optional embodiment, the target region determination module 205 includes:

[0099] The matching position determination module is used to determine the matching position of each sub-shielding area based on the matching degree of each sub-shielding area in the third tracking shielding area of ​​each group relative to each sub-shielding area in the first screen tracking shielding area of ​​the corresponding group.

[0100] The area acquisition module is used to determine the target tracking shielding area of ​​the classroom camera device based on the matching position of each shielding area.

[0101] In an optional embodiment, the matching position determination module includes calculating the similarity between each sub-shielded area in the third tracking shielding area of ​​each group and each sub-shielded area in the first screen tracking shielding area of ​​the corresponding group, determining the sub-shielded area with the highest similarity in the third tracking shielding area of ​​each group as the matching position of the corresponding sub-shielded area, and obtaining the matching position of each sub-shielded area.

[0102] It should be noted that the image tracking shielding area division device provided in the second embodiment of this application is only illustrated by the division of the above-mentioned functional modules when executing the image tracking shielding area division method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image tracking shielding area division device provided in the second embodiment of this application and the image tracking shielding area division method in the first embodiment of this application belong to the same concept, and its implementation process is detailed in the method embodiment, which will not be repeated here.

[0103] The second embodiment of the screen tracking masking area division device of this application can be applied to computer devices, such as operation and maintenance clients or recording and broadcasting devices. This device embodiment can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by the processor that processes the file reading the corresponding computer program instructions in the memory and executing them. From a hardware perspective, the computer device in which it is located may include a processor and a memory, which are interconnected through a data bus or other known means.

[0104] Please see Figure 7 This is a schematic diagram of the structure of the computer device provided in the third embodiment of this application. Figure 7 As shown, the electronic device 300 can specifically be a computer, mobile phone, tablet computer, interactive flat panel, etc. In an exemplary embodiment of this application, the electronic device 300 may include: at least one processor 310, at least one memory 320, at least one display 330, at least one network interface 340, user interface 350, and at least one communication bus 360.

[0105] The communication bus 360 is used to enable communication between these components.

[0106] The user interface 350 may include a display screen and a camera; the user interface 350 may also include standard wired and wireless interfaces.

[0107] The network interface 340 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface).

[0108] The processor 310 may include one or more processing cores. The processor 310 connects to various parts within the electronic device 300 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 320, and by calling data stored in the memory 320. Optionally, the processor 310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 310.

[0109] The memory 320 may include random access memory (RAM) or read-only memory. Optionally, the memory 320 may include a non-transitory computer-readable storage medium. The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 320 may also be at least one storage device located remotely from the aforementioned processor 310. Figure 7 As shown, the memory 320, which serves as a computer storage medium, may include an operating system, a network communication module, and a user.

[0110] exist Figure 7In the electronic device 300 shown, the user interface 350 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 310 can be used to call the application program stored in the memory 320, such as the application program of the screen tracking masking area division method; and execute the relevant operations of any screen tracking masking area division method in the above embodiments, and has the corresponding functions and beneficial effects.

[0111] The fourth embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. The instructions are adapted to be loaded by a processor and executed by the steps of the screen tracking masking region division method described above. For the specific execution process, please refer to the detailed description shown in the embodiment, which will not be repeated here. The device containing the storage medium can be an electronic device such as a personal computer, laptop computer, smartphone, or tablet computer.

[0112] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0113] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function selected in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0116] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0117] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0118] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0120] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for dividing a screen tracking masking region, characterized in that, Includes the following steps: Construct several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment; the several sets of real-world 3D images of the classroom are determined based on classroom images captured by the classroom camera equipment from several shooting angles; Image matching is performed in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms; based on the target classroom 3D reconstruction material images of each group of classrooms, several corresponding first-screen tracking and shielding areas are obtained; the classroom 3D reconstruction material image library includes 3D reconstruction images of various classroom types; Each group of classroom reality 3D images is tracked and shielded object identification to obtain several corresponding second screen tracking and shielding areas; Based on the second screen tracking shielding area of ​​each group, adjust the first screen tracking shielding area of ​​the corresponding group to obtain several groups of third screen tracking shielding areas. Based on the third-view tracking shielding area of ​​each group and the first-view tracking shielding area of ​​the corresponding group, the target tracking shielding area of ​​the classroom camera equipment is determined; wherein, each group of third-view tracking shielding areas includes several sub-shielding areas; the accuracy of determining the same shielding object corresponding to each sub-shielding area in each group of third-view tracking shielding areas is different.

2. The image tracking shielding region division method according to claim 1, characterized in that: The step of performing image matching in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms includes: The actual 3D images of the classrooms in each group are matched with the 3D reconstruction images of classrooms in the classroom 3D reconstruction material library. The 3D reconstruction image of the classroom with the highest similarity is determined as the target 3D reconstruction image of the corresponding group that matches the actual 3D images of the classrooms.

3. The image tracking shielding region division method according to claim 1, characterized in that: The step of obtaining several sets of first-screen tracking and masking areas based on the three-dimensional reconstruction material images of each set of target classrooms includes: Based on the mapping relationship between each classroom 3D reconstruction material image and the screen tracking shielding area in the classroom 3D reconstruction material image library, the first screen tracking shielding area corresponding to each group of target classroom 3D reconstruction material images is obtained.

4. The method for dividing the image tracking shielding area according to claim 1, characterized in that: The step of determining the target tracking shielding area of ​​the classroom camera equipment based on the third-view tracking shielding area of ​​each group and the first-view tracking shielding area of ​​the corresponding group includes: Based on the matching degree of each sub-shielding area in the third tracking shielding area of ​​each group relative to each sub-shielding area in the first screen tracking shielding area of ​​the corresponding group, the matching position of each sub-shielding area is determined. The target tracking shielding area of ​​the classroom camera equipment is determined based on the matching position of each shielding area.

5. The image tracking shielding region division method according to claim 4, characterized in that: The step of determining the matching position of each sub-shielding area based on the matching degree of each sub-shielding area in the third tracking shielding area of ​​each group relative to each sub-shielding area in the first screen tracking shielding area of ​​the corresponding group includes: Calculate the similarity between each sub-shielded area in the third tracking shielded area of ​​each group and each sub-shielded area in the first screen tracking shielded area of ​​the corresponding group; determine the sub-shielded area with the highest similarity in the third tracking shielded area of ​​each group as the matching position of the corresponding sub-shielded area, and obtain the matching position of each sub-shielded area.

6. The image tracking shielding region division method according to any one of claims 1 to 5, characterized in that: The steps of constructing several sets of real-world 3D images of the classroom corresponding to the classroom camera equipment include: Receive classroom environment dimensions information input by the user; Based on the classroom environment size information and classroom images captured by the classroom camera equipment from several shooting angles, several sets of real-world 3D classroom images corresponding to the classroom camera equipment are constructed.

7. The image tracking shielding region division method according to any one of claims 1 to 5, characterized in that: After determining the target tracking shielding area of ​​the classroom camera device based on the third-view tracking shielding area of ​​each group and the first-view tracking shielding area of ​​the corresponding group, the method further includes: Display the target tracking shielded area; In response to a change operation to each tracking shielding position in the target tracking shielding region, the target tracking shielding region is updated.

8. A device for dividing a screen tracking shielding area, characterized in that, include: The Reality 3D Map Construction Module is used to construct several sets of Reality 3D maps of the classroom corresponding to the classroom camera equipment. The aforementioned sets of classroom reality 3D images are determined based on classroom images captured by classroom camera equipment from several shooting angles; The first region determination module is used to perform image matching in the classroom 3D reconstruction material image library to obtain target classroom 3D reconstruction material images that match the actual 3D images of each group of classrooms; based on the target classroom 3D reconstruction material images of each group of classrooms, it obtains several corresponding first screen tracking and shielding areas; the classroom 3D reconstruction material image library includes 3D reconstruction images of various classroom types; The second region determination module is used to identify the tracking and shielding objects in each group of the classroom reality 3D map to obtain several corresponding second screen tracking and shielding regions. The third region determination module is used to adjust the first screen tracking shielding area of ​​the corresponding group according to the second screen tracking shielding area of ​​each group, so as to obtain several groups of third screen tracking shielding areas. The target area determination module is used to determine the target tracking shielding area of ​​the classroom camera device based on the third screen tracking shielding area of ​​each group and the first screen tracking shielding area of ​​the corresponding group; wherein, each group of third screen tracking shielding areas includes several sub-shielding areas; the accuracy of determining the same shielding object corresponding to each sub-shielding area in each group of third screen tracking shielding areas is different.

9. An electronic device comprising a processor and a memory; characterized in that, The memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1 to 7, which is a method for dividing the screen tracking masking region.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the screen tracking masking area division method as described in any one of claims 1 to 7.

Images