A blackboard tracking method and system
By acquiring blackboard image frames and performing color space conversion and line intersection operations, the problem of the blackboard running out of the camera or being obscured during remote teaching was solved, achieving precise positioning of the blackboard and improving the efficiency and effectiveness of remote teaching.
Patent Information
- Application Number
- CN202111584660.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-22
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-12-22
AI Technical Summary
During remote teaching, the blackboard can easily move out of the camera's view or be obscured, leading to a decrease in teaching efficiency.
By acquiring blackboard image frames, calculating the initial positions of the four corners of the blackboard, converting the RGB background color to the YCrCb color space for difference calculation, detecting the positions of the four borders of the blackboard, and combining line intersection operations to determine the real-time positions of the four corners of the blackboard, the precise positioning of the blackboard is achieved.
It efficiently and accurately positions the blackboard, supports remote teaching, saves on the cost of building multimedia facilities, and improves teaching effectiveness.
Smart Images

Figure CN114445455B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image recognition technology, and specifically to a blackboard tracking method and system. Background Art
[0002] When recording or live streaming a course, multimedia equipment is usually used to allow teachers to conduct multimedia demonstrations of teaching knowledge points, so that students can have a more intuitive and profound understanding and knowledge of the teaching content, thereby improving teaching effectiveness.
[0003] During remote teaching, the blackboard may appear out of the camera or be blocked, which greatly reduces the teaching efficiency. Summary of the invention
[0004] To this end, the embodiments of the present application provide a blackboard tracking method and system to efficiently and accurately locate the blackboard and provide technical support for remote teaching.
[0005] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:
[0006] According to a first aspect of an embodiment of the present application, a blackboard tracking method is provided, the method comprising:
[0007] Acquire a blackboard image frame, where the blackboard is square;
[0008] Calculate the initial positions of the four corners of the blackboard according to the blackboard image frame;
[0009] Calculate the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard in the above blackboard image frame;
[0010] The adjustment range of the positions of the four corners is calculated according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard.
[0011] Optionally, calculating the initial positions of the four corners of the blackboard according to the blackboard image frame includes:
[0012] Calculate the RGB background color based on the blackboard image frame;
[0013] Convert the RGB background color into a background color in a YCrCb color space, and convert the RGB image into a YCrCb image;
[0014] Performing difference calculation based on the background color of the YCrCb color space and the YCrCb image to obtain an image to be processed;
[0015] Performing horizontal and vertical detection on the image to be processed to detect the positions of the four frames of the blackboard;
[0016] A straight line intersection operation is performed based on the positions of the four frames to obtain the initial positions of the four corners of the blackboard.
[0017] Optionally, performing horizontal detection and vertical detection on the image to be processed to detect positions of four borders of the blackboard includes:
[0018] Setting a number of sampling points in the image to be processed so that each sampling point is sampled in an upward, downward, left or right direction;
[0019] If the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the border of the blackboard is detected, the coordinates of the point are recorded and the detection in this direction is stopped;
[0020] Based on the point coordinates of the frame of the blackboard in the upper, lower, left and right directions detected by the sampling points, the positions of the four frames of the blackboard are calculated.
[0021] Optionally, the calculating the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard of the blackboard image frame comprises:
[0022] The first positions of the four corners of the blackboard are detected within the set pixel ranges in the upper, lower, left and right directions of the initial positions of the four corners of the blackboard in the above-mentioned blackboard image frame.
[0023] Optionally, calculating the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard includes:
[0024] Determine a specific point of each frame of the blackboard according to the first positions of the four corners of the blackboard;
[0025] Use the sampling point to move the sampling in the up, down, left or right direction at a specific point on each border of the blackboard;
[0026] If the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the border of the blackboard is detected, and the moving distance of the sampling point is recorded;
[0027] If the moving distance is 0, the specific point of the frame is judged to be within the blackboard frame, and the adjustment range is 0;
[0028] If the moving distance is not 0, the position of the new frame is calculated according to the specific points of each frame, and then the second position of the four corners of the blackboard is obtained based on the straight line intersection operation as the real-time position of the four corners of the blackboard.
[0029] Optionally, after calculating the initial positions of the four corners of the blackboard according to the blackboard image frame, the method further includes:
[0030] Determine whether the blackboard is blocked based on the newly acquired blackboard image frame and the above blackboard image frame:
[0031] If blocked, the real-time positions of the four corners of the blackboard detected by the previous blackboard image frame are used as the first positions of the four corners of the blackboard.
[0032] Optionally, judging whether the blackboard is blocked according to the newly acquired blackboard image frame and the above blackboard image frame includes:
[0033] The grayscale average value is obtained by calculating the grayscale difference between the newly acquired blackboard image frame and the same position of the above blackboard image frame;
[0034] If the grayscale average value exceeds the set threshold, the blackboard is determined to be blocked; if it does not exceed, the blackboard is determined to be not blocked.
[0035] According to a second aspect of an embodiment of the present application, a blackboard tracking system is provided, the system comprising:
[0036] An image acquisition module is used to acquire a blackboard image frame, where the blackboard is square;
[0037] An initial position calculation module is used to calculate the initial positions of the four corners of the blackboard according to the blackboard image frame;
[0038] A position estimation module, used for calculating the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard in the above blackboard image frame;
[0039] The position determination module is used to calculate the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard.
[0040] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.
[0041] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.
[0042] In summary, the embodiment of the present application provides a blackboard tracking method and system, which obtains a blackboard image frame, wherein the blackboard is square; calculates the initial positions of the four corners of the blackboard according to the blackboard image frame; calculates the first positions of the four corners of the blackboard according to the newly obtained blackboard image frame and the initial positions of the four corners of the blackboard of the above blackboard image frame; calculates the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame, so as to determine the real-time positions of the four corners of the blackboard. The positioning of the blackboard is realized efficiently and accurately, providing technical support for remote teaching. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.
[0044] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.
[0045] Figure 1 A schematic diagram of a blackboard tracking method flow chart provided in an embodiment of the present application;
[0046] Figure 2 A flow chart of a blackboard tracking embodiment provided for an embodiment of the present application;
[0047] Figure 3 A block diagram of a blackboard tracking system provided in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown;
[0049] Figure 5 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0050] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] Multimedia demonstrations require a lot of facilities to cooperate. If the blackboard can be positioned in the most basic teaching environment, the multimedia information to be displayed can be directly displayed on the blackboard area through technical means to achieve facility-free multimedia remote teaching, which will undoubtedly greatly save the construction cost of multimedia teaching and achieve more convenient remote multimedia teaching. In order to achieve the above goals, the first thing to solve is the recognition and precise positioning of the blackboard.
[0052] The embodiment of the present application identifies the precise position of the blackboard and continuously and accurately locates the blackboard by analyzing the input image.
[0053] Figure 1 The following is a flow chart of a blackboard tracking method provided by an embodiment of the present application, including the following steps:
[0054] Step 101: Acquire a blackboard image frame, where the blackboard is square;
[0055] Step 102: Calculate the initial positions of the four corners of the blackboard according to the blackboard image frame;
[0056] Step 103: Calculate the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard in the above blackboard image frame;
[0057] Step 104: Calculate the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard.
[0058] In a possible implementation, in step 102, calculating the initial positions of the four corners of the blackboard according to the blackboard image frame includes:
[0059] The RGB background color is calculated according to the blackboard image frame; the RGB background color is converted into the background color of the YCrCb color space, and the RGB image is converted into the YCrCb image; a difference calculation is performed according to the background color of the YCrCb color space and the YCrCb image to obtain an image to be processed; a horizontal detection and a vertical detection are performed on the image to be processed to detect the positions of the four borders of the blackboard; a straight line intersection operation is performed based on the positions of the four borders to obtain the initial positions of the four corners of the blackboard.
[0060] In a possible implementation, the performing horizontal detection and vertical detection on the image to be processed to detect the positions of four frames of the blackboard includes:
[0061] A number of sampling points are set in the image to be processed so that each sampling point samples in the upward, downward, left or right direction; if the difference between the average values of the three channels of the YCrCb color space of the sampling point and the background color exceeds a set threshold, it is determined that the border of the blackboard is detected, the coordinates of the point are recorded and the detection in this direction is stopped; based on the point coordinates of the borders of the blackboard in the upward, downward, left and right directions detected by the sampling points, the positions of the four borders of the blackboard are calculated.
[0062] In a possible implementation, in step 103, the first positions of the four corners of the blackboard are calculated according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard of the blackboard image frame, including:
[0063] The first positions of the four corners of the blackboard are detected within the set pixel ranges in the upper, lower, left and right directions of the initial positions of the four corners of the blackboard in the above-mentioned blackboard image frame.
[0064] In a possible implementation, in step 104, the adjustment range of the positions of the four corners is calculated according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard, including:
[0065] Determine the specific point of each frame of the blackboard according to the first position of the four corners of the blackboard; use the sampling point to move and sample in the up, down, left or right direction of the specific point of each frame of the blackboard; if the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the frame of the blackboard is detected, and the moving distance of the sampling point is recorded; if the moving distance is 0, it is determined that the specific point of the frame belongs to the blackboard frame, and the adjustment range is 0; if the moving distance is not 0, the position of the new frame is calculated according to the specific point of each frame, and then the second position of the four corners of the blackboard is obtained based on the straight line intersection operation as the real-time position of the four corners of the blackboard.
[0066] In a possible implementation, after step 102, the method further includes:
[0067] Determine whether the blackboard is blocked based on the newly acquired blackboard image frame and the above blackboard image frame: if blocked, use the real-time positions of the four corners of the blackboard detected by the previous blackboard image frame as the first positions of the four corners of the blackboard.
[0068] In a possible implementation, determining whether the blackboard is blocked according to the newly acquired blackboard image frame and the above blackboard image frame includes:
[0069] The grayscale average value is obtained by calculating the difference in grayscale space between the newly acquired blackboard image frame and the same position of the above blackboard image frame; if the grayscale average value exceeds the set threshold, the blackboard is determined to be blocked; if not, the blackboard is determined to be not blocked.
[0070] Combine the following Figure 2 The blackboard tracking method embodiment is further explained.
[0071] On the first aspect, a preprocessing module is first used to perform preprocessing of blackboard tracking to establish a blackboard recognition and precise positioning processing flow and configure related variables; the related variables include background color storage variables, image storage variables, and four-corner coordinate storage variables.
[0072] Secondly, after reading the image, determine whether the system has been initialized. Initialize the system under the condition that the four corners of the blackboard are not blocked. The restriction here is that there must be no occlusion. If there is occlusion, reinitialization is required.
[0073] The third aspect is to calculate the initial positions of the four corners of the blackboard according to the blackboard image frame, specifically including:
[0074] First, the RGB background color is calculated according to the blackboard image frame; the RGB background color is converted into the background color of the YCrCb color space, and the RGB image is converted into a YCrCb image; the specific conversion method can be: Y=0.2990R+0.5870G+0.1140B; Cr=0.5000R-0.4187G-0.0813B+128; Cb=-0.1687R-0.3313G+0.5000B+128.
[0075] Furthermore, a difference calculation is performed based on the background color and the image in the YCrCb color space to obtain a high-quality, low-confusion image to be processed, and on this basis, blackboard boundary detection is performed; the blackboard boundary detection method includes horizontal detection and vertical detection. After performing horizontal detection and vertical detection on the image to be processed, the positions of the four frames of the blackboard can be detected.
[0076] The following is an example of the process of cross sampling of the blackboard image frame when there is no occlusion provided in the embodiment of the present application. Cross sampling is to perform horizontal detection at the position of half the height of the picture and to perform vertical detection at the position of half the width of the picture.
[0077] Taking horizontal detection as an example, two sampling points are selected from the inside of the blackboard at a certain pixel distance, and each sampling point is sampled in the upward and downward directions to detect the position of the upper and lower borders. Taking the upper border as an example, a sampling block of 5*5 pixels around the sampling point is used as the center, and the sampling block is moved upward. If the average value of the three channels of YCrCb of the sampling block has a large difference with the background color of the blackboard, for example, the difference exceeds 10, it is determined that the blackboard border is detected, the coordinates of the point are recorded, and the detection in this direction is stopped. Similarly, the coordinates of the upper border corresponding to another sampling point can be obtained. According to the principle of determining a straight line by two points, the straight line where the upper border is located can be obtained. Similarly, the straight line where the lower border is located can be obtained. Similarly, the vertical detection is performed in the same way, and the vertical detection movement direction is left and right. In this way, the straight lines where the four borders are located can be obtained, that is, the positions of the four borders of the blackboard are detected.
[0078] Then, based on the positions of the four frames, a straight line intersection operation is performed to obtain the initial positions of the four corners of the blackboard. The straight line intersection operation is to calculate the straight line intersection points using a plane coordinate system. This calculation method has the characteristics of high accuracy and strong anti-interference ability.
[0079] Fourthly, a zoom detection window mechanism is used to update the positions of the four corners of the blackboard in a small range. In order to improve the response speed of the system, the position information of the blackboard corners output by the previous frame is referenced when processing the continuously input new image frames.
[0080] Based on the YCrCb differential image, the new blackboard corner position is detected in a small range (a set pixel range above, below, left, and right of the blackboard corner detected in the previous frame, for example, 25 pixels), wherein the new image frame is converted to the YCrCb color space by the same method as in the second aspect, and then the background color is calculated to obtain the YCrCb differential image. It should be noted that if occlusion is detected, the distance information of the previous frame is used for estimation, that is, the previous position is temporarily used as the estimated position.
[0081] In the fifth aspect, in the embodiment of the present application, an occlusion detection module is used to detect whether there are other obstructions between the blackboard and the lens. The grayscale space difference calculation can be performed on the image frame of the current image region of interest and the same position of the previous image frame to evaluate the grayscale average value. If it exceeds the set threshold, it is determined that occlusion has occurred. The occlusion detection module is used to prevent tracking or detection failures due to occlusion, resulting in the output result seriously deviating from the target (i.e., the output calculated blackboard corner position deviates from the actual blackboard corner position by more than a set pixel range).
[0082] If occlusion is detected, the blackboard corner position of the previous image frame is directly used as the estimated position, and the true real-time position is calculated in the subsequent secondary detection, thereby obtaining the most accurate and stable blackboard four-corner positioning information.
[0083] The sixth aspect is the above-mentioned secondary detection and adjustment. After obtaining the positions of the four corners of the blackboard in the fourth aspect, a secondary detection is performed on several specific points, which may include the midpoint of the frame, a quarter point, and a three-quarter point, to determine whether the positions of these specific points are within the blackboard frame.
[0084] The specific judgment method can be: take the specific point of the above border as an example, select a 5*5 pixel block below the point (for example, take a pixel block of 2 pixels on the left and right of the point and 5 pixels below the point) and try to move upward. If the average value of the three channels of YCrCb of the sampling block has a large difference with the background color of the blackboard, for example, the difference exceeds 10, it is considered that the blackboard border is detected and the moving distance is recorded. If the moving distance is 0, it is considered that the position of the specific point is in the blackboard border; if the moving distance is not 0, the coordinates of the specific point are updated according to the moving distance, and the same is true for other borders.
[0085] The straight line where the new border is located is calculated based on the newly calculated specific points of the border, and then the intersection operation is performed to obtain the positions of the four corners of the blackboard, and the estimated positions of the blackboard corners calculated by the fourth aspect are replaced as the real-time positions of the four corners of the blackboard.
[0086] It can be seen that the blackboard tracking method provided in the embodiment of the present application can quickly and conveniently realize the accurate positioning of the blackboard position, and has good effects on resisting partial occlusion and preventing the blackboard corner position from jumping. The final output blackboard corner position information is accurate and smooth, which provides a feasible basis for realizing facility-free multimedia remote teaching. Both live classes and recorded classes can be realized with the help of this technology.
[0087] In summary, the embodiment of the present application provides a blackboard tracking method, by acquiring a blackboard image frame, the blackboard is square; calculating the initial positions of the four corners of the blackboard according to the blackboard image frame; calculating the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard of the above blackboard image frame; calculating the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame, so as to determine the real-time positions of the four corners of the blackboard. The positioning of the blackboard is realized efficiently and accurately, providing technical support for remote teaching.
[0088] Based on the same technical concept, the present application embodiment also provides a blackboard tracking system, such as Figure 3 As shown, the system comprises:
[0089] An image acquisition module 301 is used to acquire a blackboard image frame, where the blackboard is square;
[0090] An initial position calculation module 302 is used to calculate the initial positions of the four corners of the blackboard according to the blackboard image frame;
[0091] A position estimation module 303 is used to calculate the first positions of the four corners of the blackboard according to the newly acquired blackboard image frame and the initial positions of the four corners of the blackboard in the above blackboard image frame;
[0092] The position determination module 304 is used to calculate the adjustment range of the positions of the four corners according to the first positions of the four corners of the blackboard and the specific points of each frame to determine the real-time positions of the four corners of the blackboard.
[0093] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 4 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.
[0094] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port 203 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0095] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.
[0096] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.
[0097] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.
[0098] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 5 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.
[0099] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.
[0100] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.
[0101] It should be noted that:
[0102] The algorithm and display provided herein are not inherently related to any particular computer, virtual device or other equipment. Various general purpose devices can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of device. In addition, the application is not directed to any specific programming language. It should be understood that the content of the application described herein can be realized using various programming languages, and the description of the specific language above is to disclose the best mode of implementation of the application.
[0103] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0104] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.
[0105] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0106] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0107] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the creation device of the virtual machine according to the embodiment of the present application. The application can also be implemented as a device or device program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0108] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim that lists a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.
[0109] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A blackboard tracking method, characterized in that: The method comprises: Acquire a blackboard image frame, where the blackboard is square; Calculate the initial positions of the four corners of the blackboard according to the blackboard image frame; Detecting first positions of the four corners of the blackboard within set pixel ranges in the upper, lower, left and right directions of the initial positions of the four corners of the blackboard of the blackboard image frame; Determine a specific point of each frame of the blackboard according to the first positions of the four corners of the blackboard; Use the sampling point to move the sampling in the up, down, left or right direction at a specific point on each border of the blackboard; If the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the border of the blackboard is detected, and the moving distance of the sampling point is recorded; If the moving distance is 0, it is determined that the specific point of the frame belongs to the blackboard frame, and the adjustment range is 0; if the moving distance is not 0, the position of the new frame is calculated according to the specific point of each frame, and then the second position of the four corners of the blackboard is obtained based on the straight line intersection operation as the real-time position of the four corners of the blackboard.
2. The method according to claim 1, characterized in that The step of calculating the initial positions of the four corners of the blackboard according to the blackboard image frame includes: Calculate the RGB background color based on the blackboard image frame; Convert the RGB background color into a background color in a YCrCb color space, and convert the RGB image into a YCrCb image; Performing difference calculation based on the background color of the YCrCb color space and the YCrCb image to obtain an image to be processed; Performing horizontal and vertical detection on the image to be processed to detect the positions of the four frames of the blackboard; A straight line intersection operation is performed based on the positions of the four frames to obtain the initial positions of the four corners of the blackboard.
3. The method according to claim 2, characterized in that The step of performing horizontal and vertical detection on the image to be processed to detect the positions of four frames of the blackboard includes: Setting a number of sampling points in the image to be processed so that each sampling point is sampled in an upward, downward, left or right direction; If the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the border of the blackboard is detected, the coordinates of the point are recorded and the detection in this direction is stopped; Based on the point coordinates of the frame of the blackboard in the upper, lower, left and right directions detected by the sampling points, the positions of the four frames of the blackboard are calculated.
4. The method according to claim 1, characterized in that After calculating the initial positions of the four corners of the blackboard according to the blackboard image frame, the method further includes: Determine whether the blackboard is blocked based on the newly acquired blackboard image frame and the above blackboard image frame: If blocked, the real-time positions of the four corners of the blackboard detected by the previous blackboard image frame are used as the first positions of the four corners of the blackboard.
5. The method according to claim 4, characterized in that Judging whether the blackboard is blocked according to the newly acquired blackboard image frame and the above blackboard image frame includes: The grayscale average value is obtained by calculating the grayscale difference between the newly acquired blackboard image frame and the same position of the above blackboard image frame; If the grayscale average value exceeds the set threshold, the blackboard is determined to be blocked; if it does not exceed, the blackboard is determined to be not blocked.
6. A blackboard tracking system, characterized in that: The system comprises: An image acquisition module is used to acquire a blackboard image frame, where the blackboard is square; An initial position calculation module is used to calculate the initial positions of the four corners of the blackboard according to the blackboard image frame; A position estimation module, used for detecting the first positions of the four corners of the blackboard within the set pixel ranges in the upper, lower, left and right directions of the initial positions of the four corners of the blackboard of the blackboard image frame; The position determination module is used to determine the specific point of each frame of the blackboard according to the first position of the four corners of the blackboard; use the sampling point to move the sampling in the upper, lower, left or right direction of the specific point of each frame of the blackboard; if the difference between the average value of the three channels of the YCrCb color space of the sampling point and the background color exceeds the set threshold, it is determined that the frame of the blackboard is detected, and the moving distance of the sampling point is recorded; if the moving distance is 0, it is determined that the specific point of the frame belongs to the blackboard frame, and the adjustment range is 0; if the moving distance is not 0, the position of the new frame is calculated according to the specific point of each frame, and then the second position of the four corners of the blackboard is obtained based on the straight line intersection operation as the real-time position of the four corners of the blackboard.
7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-5.
