Laser pen-based electronic whiteboard interactive display method, device and system

Through sound and visual sensors, the speaker's voice and gestures are monitored, the laser point position is identified and the identification lines are generated, which solves the problem of difficult marking when interacting with laser pens and electronic whiteboards, and realizes accurate identification of speech content and convenient viewing of audiences.

CN118796043BActive Publication Date: 2024-11-15SHENZHEN BESTSTAR PHOTOELECTRIC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411294037.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-11-15
Estimated Expiration
2044-09-14

AI Technical Summary

Technical Problem

The existing interaction methods between laser pens and electronic whiteboards are difficult to achieve blackboard writing functions such as marking and marking, which makes it difficult for speakers to effectively annotate the PPT content, affecting the effectiveness of the speech.

Method used

Monitor the speaker's voice information through sound sensors, visual sensors monitor gestures, identify the location of laser points and start the trace mode, generate line segments based on the speech content, monitor the movement trajectory of laser points, guide the movement of laser points to the end of the target track, and generate identification lines. Implement effective identification of speech content.

Benefits of technology

It realizes effective identification of the speech content, ensures the accuracy of the logo line, is convenient for viewing of the audience, and improves the speech effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118796043B_ABST
    Figure CN118796043B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of display, and in particular to a laser pen-based electronic whiteboard interactive display method, device and system, wherein the method can guide the speaker to operate the laser pen by monitoring the speaker's voice, gestures and the position of the laser point, and then generate a marking line composed of multiple continuous marking line segments at the content of the speech; in the present application, it is possible to achieve effective marking of the speech content and ensure the accuracy of the marking line, which is convenient for the audience to watch.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of display, and in particular to an electronic whiteboard interactive display method, device and system based on a laser pen. Background Art

[0002] An electronic whiteboard is an advanced teaching and conference tool that provides interactive, collaborative and presentation functions. It combines the advantages of traditional whiteboards and modern digital technologies and is widely used in education, business, training and conference scenarios.

[0003] In some large public speeches, in order to cater to a large number of viewers, a large electronic whiteboard is often set up to display the speech PPT. Due to the large size of the electronic whiteboard, it is difficult for the speaker to indicate the content on the electronic whiteboard by body language, and a laser pen is often needed to interact with the electronic whiteboard to assist the speech. The existing interaction method between the laser pen and the electronic whiteboard can mainly achieve the effect of PPT page skipping and laser point indicating content, but it is difficult to achieve blackboard writing functions such as underlining and marking, which makes it difficult for the speaker to effectively annotate the existing PPT content, and thus it is difficult to ensure the effect of the speech. Summary of the invention

[0004] Based on this, it is necessary to provide an electronic whiteboard interactive display method, device and system based on a laser pen to address the above problems.

[0005] The embodiment of the present invention is implemented as follows: a laser pen-based electronic whiteboard interactive display method is applied to the electronic whiteboard, and the method comprises:

[0006] S1: Monitor the speaker’s voice information through the sound sensor, and then determine the speaker’s speech content;

[0007] S2: monitoring whether the speaker makes the first gesture through a visual sensor;

[0008] S3: When the speaker makes the first gesture, the initial position of the laser point on the electronic whiteboard is identified, and the trace mode is started, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0009] S4: In the trace mode, a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point;

[0010] S5: monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and removing other line segments;

[0011] S6: guide the laser point to move directly to the end point of the target extension trajectory;

[0012] S7: taking the end point as the initial position, executing steps S4 to S7 until the speaker makes a second gesture and stops the trace mode;

[0013] S8: Remove the generated moving track, and connect the extended tracks of each target to obtain a marking line.

[0014] In one embodiment, the present invention provides an electronic whiteboard interactive display device based on a laser pen, wherein a module in the electronic whiteboard interactive display device based on a laser pen is used to execute the electronic whiteboard interactive display method based on a laser pen, including:

[0015] The first monitoring module is used to monitor the speaker's voice information through a sound sensor, and then determine the speaker's speech content;

[0016] A second monitoring module, used for monitoring whether the speaker makes a first gesture through a visual sensor;

[0017] The first processing module is used to identify the initial position of the laser point on the electronic whiteboard when the speaker makes the first gesture, and start the trace mode, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0018] The second processing module is used to generate a plurality of line segments of a first set length according to the speech content with the initial position as the starting point in the trace leaving mode;

[0019] The third monitoring module is used to monitor the further movement of the laser point on the electronic whiteboard, so as to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and remove other line segments;

[0020] A third processing module is used to guide the laser point to move directly to the end point of the target extension trajectory;

[0021] A repeating module is used to execute steps S4 to S7 with the end point as the initial position until the speaker makes a second gesture to stop the trace mode;

[0022] The fourth processing module is used to remove the generated moving track and connect the extended tracks of each target to obtain a marking line.

[0023] In one embodiment, the present invention provides an electronic whiteboard interactive display system based on a laser pen, the system comprising:

[0024] Sound sensor, used to monitor the speaker's voice information;

[0025] Vision sensors to monitor the speaker’s gestures and the position of the laser dot on the electronic whiteboard;

[0026] The electronic whiteboard is connected with the sound sensor and the visual sensor, and is used to execute the electronic whiteboard interactive display method based on the laser pen.

[0027] The present invention provides an electronic whiteboard interactive display method, device and system based on a laser pen, wherein the method comprises monitoring the speech information of a speaker through a sound sensor, thereby determining the content of the speaker's speech; monitoring whether the speaker makes a first gesture through a visual sensor; when the speaker makes the first gesture, identifying the initial position of the laser point on the electronic whiteboard, and starting a trace mode; in the trace mode, generating a plurality of line segments of a first set length based on the speech content with the initial position as the starting point; monitoring the further movement of the laser point on the electronic whiteboard, so as to determine the target selected by the speaker according to the intersection ratio of the movement trajectory of the laser point and each line segment Extend the trajectory and remove other line segments; guide the laser point to move directly to the end point of the target extended trajectory; use the end point as the initial position and execute the above steps until the speaker makes a second gesture to stop the trace mode; remove the generated movement trajectory, and connect the target extended trajectories to obtain the identification line; in the present application, the speaker can be guided to operate the laser pen by monitoring the speaker's voice, gestures and the position of the laser point, and then generate an identification line composed of multiple continuous identification line segments at the content of his speech, which can not only achieve effective identification of the speech content, but also ensure the accuracy of the identification line, which is convenient for the audience to watch. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A first flow chart of an electronic whiteboard interactive display method based on a laser pen provided in one embodiment;

[0029] Figure 2 A second flow chart of an electronic whiteboard interactive display method based on a laser pen provided in one embodiment;

[0030] Figure 3 A schematic diagram of marking points of an electronic whiteboard interactive display method based on a laser pen provided in one embodiment;

[0031] Figure 4 A schematic diagram of a line segment generated in a laser pen-based electronic whiteboard interactive display method provided in an embodiment;

[0032] Figure 5 A schematic diagram of an erase mode display of an electronic whiteboard interactive display method based on a laser pen provided in an embodiment;

[0033] Figure 6 A schematic diagram of a module flow of an electronic whiteboard interactive display device based on a laser pen provided in one embodiment;

[0034] Figure 7A schematic diagram of the composition of an electronic whiteboard interactive display system based on a laser pen provided in one embodiment;

[0035] Figure 8 The figure is a block diagram of the internal structure of the control center in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script without departing from the scope of the present invention.

[0038] like Figure 1-Figure 2 As shown, in one embodiment, a laser pen-based electronic whiteboard interactive display method is proposed, which is applied to an electronic whiteboard. The method includes:

[0039] S1: Monitor the speaker’s voice information through the sound sensor, and then determine the speaker’s speech content;

[0040] S2: monitoring whether the speaker makes the first gesture through a visual sensor;

[0041] S3: When the speaker makes the first gesture, the initial position of the laser point on the electronic whiteboard is identified, and the trace mode is started, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0042] S4: In the trace mode, a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point;

[0043] S5: monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and removing other line segments;

[0044] S6: guide the laser point to move directly to the end point of the target extension trajectory;

[0045] S7: taking the end point as the initial position, executing steps S4 to S7 until the speaker makes a second gesture and stops the trace mode;

[0046] S8: Remove the generated moving track, and connect the extended tracks of each target to obtain a marking line.

[0047] In this embodiment, the method is executed in a large-sized electronic whiteboard (such as a 200-inch screen wall), and is mainly used in large-scale conferences where speakers need to use laser pens to give speeches. The electronic whiteboard is provided with a processor; a sound sensor and a visual sensor are provided in conjunction with the electronic whiteboard; the sound sensor may be a microphone used by the speaker for giving speeches, which is connected to the electronic whiteboard, and the microphone may receive the speaker's voice information and transmit the voice information to the electronic whiteboard; the visual sensor may be a camera, and the camera is installed in a position such that the camera's field of view includes both the entire display range of the electronic whiteboard and the speaker's body.

[0048] In this embodiment, a speech recognition algorithm model, such as a hidden Markov model, is preset in the electronic whiteboard. After receiving the speech information, the electronic whiteboard can process the speech information through the speech recognition algorithm model to obtain the speech content.

[0049] In this embodiment, the electronic whiteboard is preset with images of various gestures and the adjustment steps of the display mode corresponding to each gesture. For example, the first gesture is 'index finger vertically upward', which corresponds to 'entering the trace mode', and the second gesture is 'shaking the index finger', which corresponds to 'stopping the trace mode'; after the electronic whiteboard obtains the gesture image of the speaker through the visual sensor, the obtained gesture image can be compared with the preset gesture image to determine which gesture the speaker has made, and then adjust to the corresponding display mode.

[0050] In this embodiment, the electronic whiteboard can match the corresponding display content in the display content on the whiteboard according to the speech content, and the generated different line segments extend around the display content in different paths to form a mark for the display content; the number of generated line segments is determined according to the number of currently selectable paths, and is at least one; the first set length of the generated line segment is a shorter length (such as 10cm, 20cm), so that the speaker can choose the extension direction of the line segment again at the end of the line segment, and so on and so forth, so that the identification line finally determined by the speaker has more extension possibilities; in addition, the final identification line is composed of the generated line segments, and the overall line shape is relatively regular, avoiding the line shape bending and twisting caused by manual line drawing, which is more conducive to the audience's viewing.

[0051] In the present application, the speaker's voice, gestures and the position of the laser point can be monitored to guide the speaker in the operation of the laser pen, and then a marking line consisting of multiple continuous marking line segments can be generated at the content of the speech, which can not only effectively mark the content of the speech, but also ensure the accuracy of the marking line, making it easier for the audience to watch.

[0052] like Figure 3 As shown, as a preferred embodiment, after identifying the initial position of the laser point on the electronic whiteboard, it also includes:

[0053] Generate a marking point at the initial position, the size of which is the same as the size of the laser point, wherein the marking point coincides with the laser point;

[0054] Identify whether the speaker makes a third gesture;

[0055] If the speaker makes a third gesture, monitoring the execution degree of the third gesture;

[0056] The diameter of the marking point is adjusted according to the execution degree so that the thickness of the final marking line is adapted to the diameter of the marking point.

[0057] The third gesture is a gesture for moving the laser pen away from the electronic whiteboard, and the execution degree represents the distance of moving the laser pen away. The longer the distance is, the higher the execution degree is.

[0058] Adjusting the diameter of the marker point according to the execution level includes:

[0059] Identify the standoff distance of a laser pointer held by a speaker;

[0060] Determine the diameter change of the marking point according to the distance away;

[0061] Add the current diameter of the marked point to the diameter change to get the target diameter;

[0062] Adjust the diameter of the marking point to the target diameter and keep the marking point concentric with the laser point;

[0063] The diameter change of the marked point is calculated by the following formula:

[0064]

[0065] in, is the diameter change of the marked point, To keep away from the distance, The change in unit diameter corresponding to the preset unit distance.

[0066] In this embodiment, the third gesture may be an action in which the speaker rotates the elbow joint to drive the hand holding the laser pen to move from front to back with the forearm. Since the speaker faces the electronic whiteboard when marking, the speaker can move the laser pen away from the electronic whiteboard when performing the third gesture. The electronic whiteboard can identify whether the speaker makes the third gesture through a visual sensor. After the speaker makes the third gesture, the first distance can be determined according to the distance between the laser pen and the electronic whiteboard when the gesture starts, and the second distance can be determined according to the distance between the laser pen and the electronic whiteboard when the gesture is completed. The distance away can be obtained by subtracting the first distance from the second distance. The unit diameter change corresponding to the preset unit distance away can be 0.1 cm for every 1 cm away. At this time, if the distance away is 10 cm, the diameter change of the marking point is 1 cm. In this embodiment, the speaker can change the diameter of the marking point by executing the third gesture, and then change the thickness of the marking line formed later to meet the viewing needs of the audience.

[0067] like Figure 4 As shown, as a preferred embodiment, the speech content is text content obtained by voice recognition, and the generation of a plurality of line segments of a first set length based on the speech content with the initial position as the starting point includes:

[0068] Text matching between speech content and content displayed on the electronic whiteboard;

[0069] Determine the area where the successfully matched local display content is located;

[0070] Taking the initial position as the starting point, a number of line segments extending along the edge of the area are generated, wherein the line segments do not overlap, and the length of each line segment is a first set length, and the width is consistent with the diameter of the marking point.

[0071] Monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker based on the intersection ratio of the movement trajectory of the laser point and each line segment includes:

[0072] When the movement of the laser point is detected, the marking point is controlled to move accordingly, so as to keep the marking point and the laser point always concentric, and retain the path trace of the marking point, wherein the path trace represents the movement trajectory;

[0073] When the path trace reaches a second set length, the intersection ratio of the path trace and each line segment is calculated;

[0074] The line segment having the highest intersection ratio with the path trace is determined as the target extension trajectory.

[0075] The intersection ratio of the path trace with each line segment is calculated using the following formula:

[0076]

[0077] in, is the intersection ratio, is the area where the line segment overlaps with the path trace, is the total area of ​​the path trace.

[0078] In this embodiment, one or more boxes can be generated that just surround the successfully matched local display content. The area within the box can be regarded as the area where the successfully matched local display content is located, and the generated line segment is a line segment extending along the edge line of the box; the path trace is formed by connecting all the areas passed by the identification point during the movement process, and this embodiment uses the path trace to identify the movement trajectory; the second set length is shorter than the first set length, for example, the first set length is 10 cm and the second set length is 2 cm; when the speaker tries to control the laser point to move along a certain line segment, the intersection ratio of the line segment and the path trace is naturally higher, thereby determining the line segment selected by the speaker, and at this time, the end point of the line segment can be highlighted to directly guide the speaker to move the laser point to the end point to generate subsequent line segments, so that the speaker does not need to trace the entire line segment, which greatly improves the identification efficiency.

[0079] like Figure 5 As shown, as a preferred embodiment, after step S8, the following is further included:

[0080] monitoring whether the speaker makes a fourth gesture;

[0081] When the speaker makes the fourth gesture, the erasing mode is activated;

[0082] After the speaker controls the laser point to move the frame on the electronic whiteboard, a corresponding frame area is generated;

[0083] Identify the display content in the framed area, where the display content includes newly added content and initial content;

[0084] Generate interactive information to determine what the speaker needs to erase;

[0085] Delete the identified content and exit the erase mode.

[0086] Generate interactive information to determine what the speaker needs to erase including:

[0087] Generate a first dot and a second dot with a diameter larger than the laser dot next to the framed area;

[0088] An identifier of the newly added content is marked next to the first dot, wherein when the laser point falls on the identifier of the newly added content, the newly added content in the framed area is highlighted;

[0089] An identifier of the initial content is marked next to the second dot, wherein when the laser point falls on the identifier of the initial content, the initial content in the framed area is highlighted;

[0090] Monitor whether the laser point enters any dot;

[0091] When the laser point enters a dot, it is monitored whether the laser point stays in the dot for a period of time exceeding a preset period of time. If so, the content corresponding to the dot is determined as the content that the speaker needs to erase.

[0092] In this embodiment, the fourth gesture can be the speaker extending his palm and shaking it; the frame selection movement is the movement of the laser pen's moving path to form a closed frame, and the area in the closed frame is the frame selection area; the newly added content is the marking line, and the initial content is the content originally presented on the electronic whiteboard; the identifier is a preset symbol that can correspond to the content it identifies; the first dot and the second dot are equivalent to selection buttons. When the laser point falls on a dot, it indicates that the speaker wants to erase the content corresponding to the dot. In order to avoid misoperation, a preset time length (for example, 2 seconds) is set; in this embodiment, the speaker can erase any type of displayed content at any position to assist his speech and improve the speech effect.

[0093] like Figure 6 As shown, in one embodiment, a laser pen-based electronic whiteboard interactive display device is provided, and a module in the laser pen-based electronic whiteboard interactive display device is used to execute the laser pen-based electronic whiteboard interactive display method, including:

[0094] The first monitoring module is used to monitor the speaker's voice information through a sound sensor, and then determine the speaker's speech content;

[0095] A second monitoring module, used for monitoring whether the speaker makes a first gesture through a visual sensor;

[0096] The first processing module is used to identify the initial position of the laser point on the electronic whiteboard when the speaker makes the first gesture, and start the trace mode, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0097] The second processing module is used to generate a plurality of line segments of a first set length according to the speech content with the initial position as the starting point in the trace leaving mode;

[0098] The third monitoring module is used to monitor the further movement of the laser point on the electronic whiteboard, so as to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and remove other line segments;

[0099] A third processing module is used to guide the laser point to move directly to the end point of the target extension trajectory;

[0100] A repeating module is used to execute steps S4 to S7 with the end point as the initial position until the speaker makes a second gesture to stop the trace mode;

[0101] The fourth processing module is used to remove the generated moving track and connect the extended tracks of each target to obtain a marking line.

[0102] The process of each module realizing its own function in the electronic whiteboard interactive display device based on a laser pen provided in the embodiment of the present application can be specifically referred to in the foregoing Figure 1 The description of the illustrated embodiment will not be repeated here.

[0103] like Figure 7 As shown, in one embodiment, a laser pen-based electronic whiteboard interactive display system is provided, the system comprising:

[0104] Sound sensor, used to monitor the speaker's voice information;

[0105] Vision sensors to monitor the speaker’s gestures and the position of the laser dot on the electronic whiteboard;

[0106] The electronic whiteboard is connected with the sound sensor and the visual sensor, and is used to execute the electronic whiteboard interactive display method based on the laser pen.

[0107] In the present application, the electronic whiteboard cooperates with the sound sensor and the visual sensor, and can guide the speaker to operate the laser pen by monitoring the speaker's voice, gestures and the position of the laser point, and then generate a marking line composed of multiple continuous marking line segments at the content of the speech, which can not only effectively mark the speech content, but also ensure the accuracy of the marking line, making it easier for the audience to watch.

[0108] Figure 8 FIG. 2 shows an internal structure diagram of a computer device in one embodiment. Figure 8As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the electronic whiteboard interactive display method based on a laser pen provided in an embodiment of the present invention. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the electronic whiteboard interactive display method based on a laser pen provided in an embodiment of the present invention. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0109] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, the electronic whiteboard interactive display device based on the laser pen provided in the embodiment of the present invention can be implemented in the form of a computer program. The computer program can be used in Figure 8 The computer device shown in the figure is run on the computer device. The memory of the computer device can store various program modules constituting the electronic whiteboard interactive display device based on the laser pen, for example, Figure 6 The computer program composed of the first monitoring module, the second monitoring module, the first processing module, the second processing module, the third monitoring module, the third processing module, the repeating module, and the fourth processing module shown in the figure enables the processor to execute the steps of the electronic whiteboard interactive display method based on a laser pen in each embodiment of the present invention described in this specification.

[0111] For example, Figure 8 The computer device shown can be Figure 6 The first monitoring module in the laser pen-based electronic whiteboard interactive display device shown executes step S1; the computer device can execute step S2 through the second monitoring module; the computer device can execute step S3 through the first processing module; the computer device can execute step S4 through the second processing module; the computer device can execute step S5 through the third monitoring module; the computer device can execute step S4 through the second processing module; the computer device can execute step S4 through the second processing module; the computer device can execute step S4 through the second processing module.

[0112] In one embodiment, a computer device is provided, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0113] S1: Monitor the speaker’s voice information through the sound sensor, and then determine the speaker’s speech content;

[0114] S2: monitoring whether the speaker makes the first gesture through a visual sensor;

[0115] S3: When the speaker makes the first gesture, the initial position of the laser point on the electronic whiteboard is identified, and the trace mode is started, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0116] S4: In the trace mode, a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point;

[0117] S5: monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and removing other line segments;

[0118] S6: guide the laser point to move directly to the end point of the target extension trajectory;

[0119] S7: taking the end point as the initial position, executing steps S4 to S7 until the speaker makes a second gesture and stops the trace mode;

[0120] S8: Remove the generated moving track and connect the extended tracks of each target to obtain a marking line. In one embodiment, a computer readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following steps:

[0121] S1: Monitor the speaker’s voice information through the sound sensor, and then determine the speaker’s speech content;

[0122] S2: monitoring whether the speaker makes the first gesture through a visual sensor;

[0123] S3: When the speaker makes the first gesture, the initial position of the laser point on the electronic whiteboard is identified, and the trace mode is started, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard;

[0124] S4: In the trace mode, a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point;

[0125] S5: monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and removing other line segments;

[0126] S6: guide the laser point to move directly to the end point of the target extension trajectory;

[0127] S7: taking the end point as the initial position, executing steps S4 to S7 until the speaker makes a second gesture and stops the trace mode;

[0128] S8: Remove the generated moving track, and connect the extended tracks of each target to obtain a marking line.

[0129] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0130] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0131] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An electronic whiteboard interactive display method based on a laser pen, applied to an electronic whiteboard, characterized in that: The method comprises: S1: Monitor the speaker’s voice information through the sound sensor, and then determine the speaker’s speech content; S2: monitoring whether the speaker makes the first gesture through a visual sensor; S3: When the speaker makes the first gesture, the initial position of the laser point on the electronic whiteboard is identified, and the trace mode is started, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard; S4: In the trace mode, a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point; S5: monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and removing other line segments; S6: guide the laser point to move directly to the end point of the target extension trajectory; S7: taking the end point as the initial position, executing steps S4 to S7 until the speaker makes a second gesture and stops the trace mode; S8: Remove the generated moving track, and connect the extended tracks of each target to obtain a marking line.

2. The method according to claim 1, characterized in that After identifying the initial position of the laser point on the electronic whiteboard, it also includes: Generate a marking point at the initial position, the size of which is the same as the size of the laser point, wherein the marking point coincides with the laser point; Identify whether the speaker makes a third gesture; If the speaker makes a third gesture, monitoring the execution degree of the third gesture; The diameter of the marking point is adjusted according to the execution degree so that the thickness of the final marking line is adapted to the diameter of the marking point.

3. The method according to claim 2, characterized in that The third gesture is a gesture for moving the laser pen away from the electronic whiteboard, and the execution degree represents the distance of moving the laser pen away. The longer the distance is, the higher the execution degree is. Adjusting the diameter of the marker point according to the execution level includes: Identify the standoff distance of a laser pointer held by a speaker; Determine the diameter change of the marking point according to the distance away; Add the current diameter of the marked point to the diameter change to get the target diameter; Adjust the diameter of the marking point to the target diameter and keep the marking point concentric with the laser point; The diameter change of the marked point is calculated by the following formula: in, is the diameter change of the marked point, To keep away from the distance, The change in unit diameter corresponding to the preset unit distance.

4. The method according to claim 2, characterized in that: The speech content is text content obtained through speech recognition, and a plurality of line segments of a first set length are generated based on the speech content with the initial position as the starting point, including: Text matching between speech content and content displayed on the electronic whiteboard; Determine the area where the successfully matched local display content is located; Taking the initial position as the starting point, a number of line segments extending along the edge of the area are generated, wherein the line segments do not overlap, and the length of each line segment is a first set length, and the width is consistent with the diameter of the marking point.

5. The method according to claim 4, characterized in that Monitoring the further movement of the laser point on the electronic whiteboard to determine the target extension trajectory selected by the speaker based on the intersection ratio of the movement trajectory of the laser point and each line segment includes: When the movement of the laser point is detected, the marking point is controlled to move accordingly, so as to keep the marking point and the laser point always concentric, and retain the path trace of the marking point, wherein the path trace represents the movement trajectory; When the path trace reaches a second set length, the intersection ratio of the path trace and each line segment is calculated; The line segment having the highest intersection ratio with the path trace is determined as the target extension trajectory.

6. The method according to claim 5, characterized in that The intersection ratio of the path trace with each line segment is calculated using the following formula: in, is the intersection ratio, is the area where the line segment overlaps with the path trace, is the total area of ​​the path trace.

7. The method according to claim 1, characterized in that After step S8, the method further includes: monitoring whether the speaker makes a fourth gesture; When the speaker makes the fourth gesture, the erasing mode is activated; After the speaker controls the laser point to move the frame on the electronic whiteboard, a corresponding frame area is generated; Identify the display content in the framed area, where the display content includes newly added content and initial content; Generate interactive information to determine what the speaker needs to erase; Delete the identified content and exit the erase mode.

8. The method according to claim 7, characterized in that Generate interactive information to determine what the speaker needs to erase including: Generate a first dot and a second dot with a diameter larger than the laser dot next to the framed area; An identifier of the newly added content is marked next to the first dot, wherein when the laser point falls on the identifier of the newly added content, the newly added content in the framed area is highlighted; An identifier of the initial content is marked next to the second dot, wherein when the laser point falls on the identifier of the initial content, the initial content in the framed area is highlighted; Monitor whether the laser point enters any dot; When the laser point enters a dot, it is monitored whether the laser point stays in the dot for a period of time exceeding a preset period of time. If so, the content corresponding to the dot is determined as the content that the speaker needs to erase.

9. An electronic whiteboard interactive display device based on a laser pen, characterized in that: The module in the laser pen-based electronic whiteboard interactive display device is used to execute the laser pen-based electronic whiteboard interactive display method according to claim 1, including: The first monitoring module is used to monitor the speaker's voice information through a sound sensor, and then determine the speaker's speech content; A second monitoring module, used for monitoring whether the speaker makes a first gesture through a visual sensor; The first processing module is used to identify the initial position of the laser point on the electronic whiteboard when the speaker makes the first gesture, and start the trace mode, wherein in the trace mode, the movement track of the laser point is displayed on the electronic whiteboard; The second processing module is used to generate a plurality of line segments of a first set length according to the speech content with the initial position as the starting point in the trace leaving mode; The third monitoring module is used to monitor the further movement of the laser point on the electronic whiteboard, so as to determine the target extension trajectory selected by the speaker according to the intersection ratio between the movement trajectory of the laser point and each line segment, and remove other line segments; A third processing module is used to guide the laser point to move directly to the end point of the target extension trajectory; A repeating module is used to execute steps S4 to S7 with the end point as the initial position until the speaker makes a second gesture to stop the trace mode; The fourth processing module is used to remove the generated moving track and connect the extended tracks of each target to obtain a marking line.

10. An electronic whiteboard interactive display system based on a laser pen, characterized in that: The system comprises: Sound sensor, used to monitor the speaker's voice information; Vision sensors to monitor the speaker’s gestures and the position of the laser dot on the electronic whiteboard; An electronic whiteboard is connected to a sound sensor and a visual sensor, and is used to execute the electronic whiteboard interactive display method based on a laser pen as described in any one of claims 1 to 8.