Circling processing method and apparatus based on electronic whiteboard, and device and storage medium

Abstract

Provided in the present invention are a circling processing method and apparatus based on an electronic whiteboard, and a device and a storage medium. The method comprises: capturing a touch event on the surface of an electronic whiteboard and recognizing the touch event, and generating a circling area on the basis of a recognition result; performing object recognition on the circling area to acquire a recognized object in the circling area, and calling a corresponding operation menu on the surface of the electronic whiteboard on the basis of the type of the recognized object, wherein the operation menu comprises several operation options; and in response to an operation instruction sent by a user on the operation menu, performing object processing on the recognized object. The present invention can optimize a circling function processing flow of an electronic whiteboard large-screen all-in-one machine, thereby improving user operation experience.

Description

Method, apparatus, equipment and storage medium for selection processing based on electronic whiteboard Technical Field

[0001] This invention relates to the field of electronic whiteboard large screen all-in-one machine technology, and in particular to a selection processing method, device, equipment and storage medium based on electronic whiteboard. Background Technology

[0002] An all-in-one electronic whiteboard is an interactive device that integrates a large-screen display, touch functionality, and a computer system. It combines the functions of a traditional whiteboard with modern display and touch technology, improving the efficiency of information presentation and communication through large-screen display, touch operation, and multimedia functions. It provides users with a more efficient and convenient operating experience and is commonly used in education, business, and conference settings.

[0003] While large-screen interactive whiteboards offer multi-touch and handwriting capabilities, they may exhibit errors in recognizing user selections and intentions, particularly when processing handwritten text and inserted images. The lack of convenient selection and operation functions fails to meet diverse user needs. Users frequently require editing, searching, deleting, and copying written content and images, but existing technologies do not provide efficient and intuitive implementation methods. Furthermore, during selection, users may lack real-time feedback, unsure if their selection is correct. Therefore, existing large-screen interactive whiteboards suffer from inconvenient operation and insufficient feedback in their selection and operation functions, negatively impacting the user experience. Summary of the Invention

[0004] The present invention aims to provide a selection processing method, apparatus, device and storage medium based on electronic whiteboard, so as to solve the above-mentioned technical problems, optimize the selection function processing flow of electronic whiteboard large screen all-in-one machine, and improve the user operation experience.

[0005] To address the aforementioned technical problems, this invention provides a selection processing method based on an electronic whiteboard, comprising:

[0006] The system captures and identifies touch events on the surface of the electronic whiteboard, and generates a selected area based on the identification results.

[0007] The selected area is used to identify objects, and the identified objects within the selected area are obtained. The corresponding operation menu is called on the surface of the electronic whiteboard according to the type of the identified objects. The operation menu includes several operation options.

[0008] The identified object is processed in response to the operation command sent by the user on the operation menu.

[0009] In the above solution, touch events on the electronic whiteboard surface are captured, and the area selected by the user is generated through touch event recognition. The system can capture the user's selection action in real time, allowing the user to immediately see the selected area, thus reducing the probability of misselection. Touch event capture and recognition ensure the accuracy of the selected area, contributing to improved accuracy of subsequent operations. The content within the selected area is identified, and objects within the area are determined. Based on the type of the identified object, a corresponding operation menu is generated. The operation options in the menu are related to the selected object, making the operation process more intuitive and convenient. Users can complete operations with simple clicks, without needing to search for other menus or toolbars. By calling different operation menus, users can perform diverse operations on different types of content, meeting various usage scenarios. The instant display of the menu and the intuitive presentation of operation options enable users to better understand and use the electronic whiteboard's functions. Based on the user's instructions sent in the operation menu, the identified objects within the selected area are processed accordingly, enabling rapid response to user operation commands and timely object processing, avoiding cumbersome operation steps. Furthermore, the direct relationship between the operation menu and the selected object reduces the risk of content modification or loss due to accidental operation.

[0010] In one implementation, the step of capturing touch events on the surface of the electronic whiteboard, identifying the touch events, and generating a selection area based on the identification result specifically includes:

[0011] The touch events on the electronic whiteboard surface are monitored in real time based on the touch screen driver; wherein the touch events include touch start events, touch movement events, and touch end events;

[0012] Each time a touch event is detected, a touch point location information is generated.

[0013] When the user stays on the electronic whiteboard surface for more than a first preset time, it is determined that the touch end event has been detected and the electronic whiteboard is controlled to enter the selection mode.

[0014] The selected area is generated on the surface of the electronic whiteboard based on the accumulated first contact information.

[0015] In one implementation, the step of capturing touch events on the surface of the electronic whiteboard, identifying the touch events, and generating a selection area based on the identification result further includes:

[0016] Gesture information is obtained by capturing touch events on the surface of the electronic whiteboard based on a gesture recognition library;

[0017] The boundary of the selected area is calculated based on the gesture information and the boundary is generated on the surface of the electronic whiteboard to obtain the selected area.

[0018] In one implementation, the step of acquiring the identified objects within the selected area and calling the corresponding operation menu on the electronic whiteboard surface according to the type of the identified objects specifically includes:

[0019] Obtain the objects to be identified within the selected area; wherein, the types of the objects to be identified include: images and text;

[0020] When the type of the object to be identified is an image, a first floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the first floating menu include: image search, image deletion, image copy, and text selection;

[0021] When the type of the identified object is text, a second floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the second floating menu include: font conversion, text search, text deletion and text copy.

[0022] In one implementation, when the type of the identified object is text, before generating the second floating menu on the electronic whiteboard surface, the following steps are also included:

[0023] Handwriting detection is performed on the text within the selected area. When the text is handwritten, it is converted into standard text based on OCR technology; wherein the standard text is editable text.

[0024] In one implementation, the object processing of the identified object in response to an operation command sent by the user on the operation menu specifically includes:

[0025] When the operation instruction is an image search instruction, the first preset interface is called to perform an image search and the image search results are displayed on the surface of the electronic whiteboard.

[0026] When the operation instruction is an image deletion instruction, the image within the selected area is deleted and the electronic whiteboard surface is re-rendered;

[0027] When the operation instruction is an image copy instruction, the image within the selected area is copied to a preset location;

[0028] When the operation instruction is to select text, the text contained in the image within the selected area is extracted and copied to the preset location.

[0029] In one implementation, the step of performing object processing on the identified object in response to an operation command sent by a user on the operation menu further includes:

[0030] When the operation instruction is a font conversion instruction, the text within the selected area is converted into a preset font based on OCR technology, and the preset font is re-rendered on the surface of the electronic whiteboard.

[0031] When the operation instruction is a text search instruction, the second preset interface is invoked to perform a text search and the text search results are displayed on the surface of the electronic whiteboard.

[0032] When the operation instruction is a text deletion instruction, the text within the selected area is deleted and the electronic whiteboard surface is re-rendered;

[0033] When the operation instruction is a text copy instruction, the text within the selected area is copied to a preset position.

[0034] Secondly, this application also provides a selection processing device based on an electronic whiteboard, including: a touch recognition module, a menu generation module, and an instruction response module;

[0035] The touch recognition module is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selected area based on the recognition results;

[0036] The menu generation module is used to perform object recognition on the selected area, obtain the recognized objects within the selected area, and call the corresponding operation menu on the electronic whiteboard surface according to the type of the recognized objects; wherein, the operation menu includes several operation options;

[0037] The instruction response module is used to perform object processing on the identified object in response to the operation instructions sent by the user on the operation menu.

[0038] In the above solution, touch events on the electronic whiteboard surface are captured, and the area selected by the user is generated through touch event recognition. The system can capture the user's selection action in real time, allowing the user to immediately see the selected area, thus reducing the probability of misselection. Touch event capture and recognition ensure the accuracy of the selected area, contributing to improved accuracy of subsequent operations. The content within the selected area is identified, and objects within the area are determined. Based on the type of the identified object, a corresponding operation menu is generated. The operation options in the menu are related to the selected object, making the operation process more intuitive and convenient. Users can complete operations with simple clicks, without needing to search for other menus or toolbars. By calling different operation menus, users can perform diverse operations on different types of content, meeting various usage scenarios. The instant display of the menu and the intuitive presentation of operation options enable users to better understand and use the electronic whiteboard's functions. Based on the user's instructions sent in the operation menu, the identified objects within the selected area are processed accordingly, enabling rapid response to user operation commands and timely object processing, avoiding cumbersome operation steps. Furthermore, the direct relationship between the operation menu and the selected object reduces the risk of content modification or loss due to accidental operation.

[0039] In one implementation, the touch recognition module is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, generating a selected area based on the recognition result, specifically including:

[0040] The touch events on the electronic whiteboard surface are monitored in real time based on the touch screen driver; wherein the touch events include touch start events, touch movement events, and touch end events;

[0041] Each time a touch event is detected, a touch point location information is generated.

[0042] When the user stays on the electronic whiteboard surface for more than a first preset time, it is determined that the touch end event has been detected and the electronic whiteboard is controlled to enter the selection mode.

[0043] The selected area is generated on the surface of the electronic whiteboard based on the accumulated first contact information.

[0044] In one implementation, the touch recognition module is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selected area based on the recognition result, and further includes:

[0045] Gesture information is obtained by capturing touch events on the surface of the electronic whiteboard based on a gesture recognition library;

[0046] The boundary of the selected area is calculated based on the gesture information and the boundary is generated on the surface of the electronic whiteboard to obtain the selected area.

[0047] In one implementation, the step of acquiring the identified objects within the selected area and calling the corresponding operation menu on the electronic whiteboard surface according to the type of the identified objects specifically includes:

[0048] Obtain the objects to be identified within the selected area; wherein, the types of the objects to be identified include: images and text;

[0049] When the type of the object to be identified is an image, a first floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the first floating menu include: image search, image deletion, image copy, and text selection;

[0050] When the type of the identified object is text, a second floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the second floating menu include: font conversion, text search, text deletion and text copy.

[0051] In one implementation, when the type of the identified object is text, before generating the second floating menu on the electronic whiteboard surface, the following steps are also included:

[0052] Handwriting detection is performed on the text within the selected area. When the text is handwritten, it is converted into standard text based on OCR technology; wherein the standard text is editable text.

[0053] In one implementation, the instruction response module is used to perform object processing on the identified object in response to an operation instruction sent by the user on the operation menu, specifically including:

[0054] When the operation instruction is an image search instruction, the first preset interface is called to perform an image search and the image search results are displayed on the surface of the electronic whiteboard.

[0055] When the operation instruction is an image deletion instruction, the image within the selected area is deleted and the electronic whiteboard surface is re-rendered;

[0056] When the operation instruction is an image copy instruction, the image within the selected area is copied to a preset location;

[0057] When the operation instruction is to select text, the text contained in the image within the selected area is extracted and copied to the preset location.

[0058] In one implementation, the instruction response module is used to perform object processing on the identified object in response to an operation instruction sent by the user on the operation menu, and further includes:

[0059] When the operation instruction is a font conversion instruction, the text within the selected area is converted into a preset font based on OCR technology, and the preset font is re-rendered on the surface of the electronic whiteboard.

[0060] When the operation instruction is a text search instruction, the second preset interface is invoked to perform a text search and the text search results are displayed on the surface of the electronic whiteboard.

[0061] When the operation instruction is a text deletion instruction, the text within the selected area is deleted and the electronic whiteboard surface is re-rendered;

[0062] When the operation instruction is a text copy instruction, the text within the selected area is copied to a preset position.

[0063] Thirdly, this application also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the electronic whiteboard-based selection processing method described above.

[0064] Fourthly, this application also provides a computer-readable storage medium, which includes a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to perform the electronic whiteboard-based selection processing method as described above. Attached Figure Description

[0065] Figure 1 is a flowchart illustrating a selection process based on an electronic whiteboard according to an embodiment of the present invention.

[0066] Figure 2 is a structural diagram of a selection processing device based on an electronic whiteboard provided in one embodiment of the present invention. Detailed Implementation

[0067] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0068] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0069] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0070] First, some of the terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0071] (1) Selection: In electronic whiteboard or touch screen applications, "selection" refers to the user drawing a closed area on the screen with their finger or stylus to select or manipulate the content within that area.

[0072] Example 1

[0073] Referring to Figure 1, Figure 1 is a flowchart illustrating a whiteboard-based selection method according to an embodiment of the present invention. This embodiment of the present invention provides a whiteboard-based selection method, including steps 101 to 103, each step of which is detailed below:

[0074] Step 101: Capture touch events on the surface of the electronic whiteboard and identify the touch events, and generate a selected area based on the identification results.

[0075] In one embodiment, capturing and identifying touch events on the electronic whiteboard surface, and generating a selection area based on the identification results, specifically includes: real-time monitoring of touch events on the electronic whiteboard surface based on a touchscreen driver; wherein the touch events include touch start events, touch movement events, and touch end events; generating touch point position information each time a touch event is detected; determining that the touch end event has been detected and controlling the electronic whiteboard to enter the selection mode when the user stays on the electronic whiteboard surface for more than a first preset time; generating the selection area based on the accumulated first touch point information. Electronic whiteboards are typically equipped with touchscreen technology, so their touch events can be captured and processed by touchscreen drivers. These drivers are responsible for converting the raw signals from the touchscreen into events that the operating system or application can understand. In this embodiment of the invention, real-time monitoring of touch events on the electronic whiteboard surface based on a touchscreen driver includes touch start events, touch movement events, and touch end events. Different operating systems and whiteboard brands may use different touchscreen drivers and APIs. A suitable touchscreen driver should be selected according to the operating system of the electronic whiteboard device. For example, Windows touchscreen drivers typically use the WM_TOUCH message to capture touch events; Android systems capture touch events through the MotionEvent class; and Linux systems can handle touch events through the evdev or libinput libraries. When a user touches the screen, the system generates a touch start event. This event typically includes information such as the location and ID of the touch point. When a user moves their finger or stylus on the screen, the system generates a touch movement event. This event typically includes the location of the current touch point. When the user leaves the screen, the system generates a touch end event. This event typically includes the location and ID of the last touch point. For example, in this embodiment of the invention, the system identifies the action of the user's finger or stylus stopping on the electronic whiteboard. When it detects that the user's finger or stylus has stopped on the whiteboard surface for more than 2 seconds, it enters a selection mode. Using the captured touch point coordinates, a rectangular or other shaped selection area can be calculated, and the calculated selection area is drawn on the screen as visual feedback to the user.

[0076] In one embodiment, as an alternative to the embodiments of the present invention, touch events on the electronic whiteboard surface can also be captured through gesture recognition. Android systems can use classes such as `GestureDetector` and `ScaleGestureDetector` to detect gestures. Windows systems can use the `Windows.UI.Input.GestureRecognizer` class (in UWP applications) or other related libraries. Cross-platform libraries such as TensorFlow.js's gesture recognition model are suitable for web applications. Gesture recognition libraries can identify common gestures (such as selection, clicking, swiping, etc.) to obtain the user's gesture information, i.e., the coordinates of the touch point. Based on the touch point coordinates obtained after a gesture, i.e., the gesture information, the coordinates of the upper left and lower right corners of the selected area are calculated and a rectangle or other shape is drawn on the electronic whiteboard to represent the selected area.

[0077] Step 102: Perform object recognition on the selected area to obtain the recognized objects within the selected area, and call the corresponding operation menu on the electronic whiteboard surface according to the type of the recognized objects; wherein, the operation menu includes several operation options.

[0078] In this embodiment of the invention, image processing techniques are used to distinguish text from images within a selected area. Common techniques include color detection and shape detection. Color detection is a technique that distinguishes handwriting from background images by identifying regions of different colors in an image, suitable for scenarios with significant color contrast. For example, the image is converted to different color spaces (such as HSV, Lab) to make it easier to separate the colors of interest. For instance, converting the image from RGB to HSV makes color separation more intuitive; a color range is set to detect the color of the handwriting. For example, in the HSV space, the range of the H, S, and V channels can be set to filter specific colors; a binary image is generated using a color threshold, where the handwriting area is divided into white or black (or other colors), and the background area is excluded. Shape detection techniques are used to identify and extract regions with specific geometric shapes in an image. They are commonly used to detect the contours of handwriting. For example, the image is converted to a grayscale image and smoothed to reduce noise; edge detection algorithms (such as Canny edge detection) are used to extract the edges of the handwriting; dilation and erosion operations are applied to enhance the shape features of the handwriting; and contour detection algorithms are used to identify and extract the contours of the handwriting.

[0079] In one embodiment, the step of acquiring the identification object within the selected area and calling the corresponding operation menu on the electronic whiteboard surface according to the type of the identification object specifically includes: acquiring the identification object within the selected area; wherein the type of the identification object includes: image and text; when the type of the identification object is an image, a first floating menu is generated on the electronic whiteboard surface; wherein the operation options of the first floating menu include: image search, image deletion, image copy, and text selection; when the type of the identification object is text, a second floating menu is generated on the electronic whiteboard surface; wherein the operation options of the second floating menu include: font conversion, text search, text deletion, and text copy.

[0080] In this embodiment of the invention, floating menus corresponding to different recognition objects are pre-generated within the control system of the electronic whiteboard. These menus are then invoked based on the different object types within the selected area, generating different floating menus on the electronic whiteboard surface. These floating menus allow users to quickly perform related operations without leaving the current operation. Users do not need to manually search for functions in menus or toolbars, significantly improving operational efficiency and workflow smoothness. The floating menus can dynamically adjust their displayed content based on the selected content or position, providing immediate feedback to help users complete operations more accurately. Furthermore, the operation options in the floating menus for different recognition objects also differ, providing relevant operation options based on the content of the selected area, making operations more convenient and improving the user experience.

[0081] In one embodiment, when the type of the object to be identified is text, before generating the second floating menu on the electronic whiteboard surface, the method further includes: performing handwriting detection on the text within the selected area; when the text is handwritten, converting the handwritten text into standard text based on OCR technology; wherein the standard text is editable text.

[0082] In this embodiment of the invention, handwriting detection is performed through edge detection algorithms, contour analysis, and handwriting feature extraction. Edge detection algorithms (such as Canny edge detection) are used to identify the edges of handwriting in an image; handwriting typically exhibits distinct features at its edges. Contour analysis detects contours in the image; the contours of handwriting are usually continuous and regular. Handwriting features are extracted, such as line thickness and continuity; handwriting typically exhibits consistent characteristics. Preferably, to improve the accuracy of handwriting detection, preprocessing operations such as image cropping, noise reduction, and binarization can be performed on the selected area. When the selected area contains handwriting, it can be converted into standard text using an open-source OCR library (such as Tesseract) or by integrating a third-party OCR API. For example, Tesseract can recognize text in multiple languages ​​and supports handwritten text recognition; third-party OCR APIs such as Google Cloud Vision API and Microsoft Azure Computer Vision typically offer stronger handwriting recognition capabilities and higher accuracy. OCR technology converts handwriting into standard editable text. This step includes recognizing and classifying each character to generate the corresponding text. The recognized text is returned to the user and displayed on the electronic whiteboard, where the user can further edit or use the text.

[0083] Step 103: Perform object processing on the identified object in response to the operation command sent by the user on the operation menu.

[0084] In one embodiment, the object processing in response to the operation command sent by the user on the operation menu specifically includes: when the operation command is an image search command, calling a first preset interface to perform an image search and displaying the image search results on the electronic whiteboard surface; when the operation command is an image deletion command, deleting the image within the selected area and re-rendering the electronic whiteboard surface; when the operation command is an image copy command, copying the image within the selected area to a preset position; and when the operation command is a text selection command, extracting the text contained in the image within the selected area and copying it to the preset position.

[0085] In this embodiment of the invention, when a user clicks the "Image Search" option, the system calls the Google Image Search API to display relevant image search results; when the user clicks the "Image Delete" option, the selected image is deleted and the whiteboard content is re-rendered; when the user clicks the "Image Copy" option, the selected image content is copied to the clipboard for other applications to paste; when the user clicks the "Select Text" option, if the image within the selected area contains text, the text content in the image is detected using OCR technology. After the user clicks "Select Text," the text content is extracted and copied for pasting. As a preferred embodiment of the invention, the system displays the search results in a multi-window format on the right side of the screen for easy viewing by the user.

[0086] In one embodiment, the step of processing the identified object in response to an operation command sent by the user on the operation menu further includes: when the operation command is a font conversion command, converting the text within the selected area into a preset font based on OCR technology and re-rendering the preset font on the electronic whiteboard surface; when the operation command is a text search command, calling a second preset interface to perform a text search and displaying the text search results on the electronic whiteboard surface; when the operation command is a text deletion command, deleting the text within the selected area and re-rendering the electronic whiteboard surface; and when the operation command is a text copy command, copying the text within the selected area to a preset position.

[0087] In this embodiment of the invention, when the user clicks the "Font Conversion" option, the OCR module pre-integrated in the system is invoked to convert the handwriting into a standard font. The converted text replaces the handwriting and is then re-rendered on the whiteboard. The specific implementation process of converting handwriting into a standard font using OCR technology has been described in detail above and will not be repeated here. When the user clicks the "Text Search" option, a browser or search engine API is invoked to perform a search, and the search results are displayed in a multi-window manner on the right side of the screen. When the user clicks the "Text Delete" option, the text within the selected area is deleted and the whiteboard content is re-rendered. When the user clicks the "Text Copy" option, the text within the selected area is copied to the clipboard for other applications to paste.

[0088] In this embodiment of the invention, a whiteboard-based selection processing device is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-described whiteboard-based selection processing method.

[0089] In this embodiment of the invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the above-described electronic whiteboard-based selection processing method when it is running.

[0090] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to perform the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in an electronic whiteboard-based selection processing device.

[0091] The interactive whiteboard-based selection processing device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The interactive whiteboard-based selection processing device may include, but is not limited to, a processor, memory, and a display. Those skilled in the art will understand that the above components are merely examples of the interactive whiteboard-based selection processing device and do not constitute a limitation on the device. It may include more or fewer components, combinations of certain components, or different components. For example, the interactive whiteboard-based selection processing device may also include input / output devices, network access devices, buses, etc.

[0092] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic whiteboard-based selection processing device, connecting all parts of the device via various interfaces and lines.

[0093] The memory can be used to store computer programs and / or modules. The processor implements various functions of the electronic whiteboard-based selection processing device by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0094] In this invention, the module for selecting elements on an electronic whiteboard, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Those skilled in the art can understand and implement this invention without any inventive effort.

[0095] This invention provides a selection processing method based on an electronic whiteboard. It captures user touch events on the whiteboard surface and generates the user-selected area by recognizing these touch events. This real-time capture of the user's selection action allows the user to immediately see the selected area, reducing the probability of incorrect selection. The capture and recognition of touch events ensures the accuracy of the selected area, improving the accuracy of subsequent operations. The method identifies the content within the selected area, determines the objects within the area, and generates corresponding operation menus based on the type of the identified objects. The operation options in the menus are related to the selected objects, making the operation process more intuitive and convenient. Users can complete operations with simple clicks without needing to search for other menus or toolbars. By calling different operation menus, users can perform diverse operations on different types of content, meeting various usage scenarios. The instant display of the menus and the intuitive presentation of operation options enable users to better understand and use the electronic whiteboard's functions. Based on the user's instructions sent in the operation menu, the method processes the identified objects within the selected area accordingly, quickly responding to user operation commands and promptly processing objects, avoiding cumbersome operation steps. Furthermore, the operation menu is directly related to the selected object, which can reduce the risk of content modification or loss due to accidental operation.

[0096] Example 2

[0097] Referring to Figure 2, Figure 2 is a structural diagram of a whiteboard-based selection processing device provided in an embodiment of the present invention. The embodiment of the present invention provides a whiteboard-based selection processing device, including: a touch recognition module 201, a menu generation module 202, and an instruction response module 203;

[0098] The touch recognition module 201 is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selected area based on the recognition results;

[0099] The menu generation module 202 is used to perform object recognition on the selected area, obtain the recognized objects within the selected area, and call the corresponding operation menu on the electronic whiteboard surface according to the type of the recognized objects; wherein, the operation menu includes several operation options;

[0100] The instruction response module 203 is used to perform object processing on the identified object in response to the operation instruction sent by the user on the operation menu.

[0101] In one embodiment, the touch recognition module 201 is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selection area based on the recognition results. Specifically, it includes: monitoring touch events on the surface of the electronic whiteboard in real time based on the touch screen driver; wherein, the touch events include touch start events, touch movement events, and touch end events; generating touch point position information each time a touch event is detected; when the user stays on the surface of the electronic whiteboard for more than a first preset time, determining that the touch end event has been detected and controlling the electronic whiteboard to enter the selection mode; and generating the selection area on the surface of the electronic whiteboard based on the accumulated first touch point information.

[0102] In one embodiment, the touch recognition module 201 is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selected area based on the recognition result. It further includes: capturing touch events on the surface of the electronic whiteboard based on a gesture recognition library to obtain gesture information; calculating the boundary of the selected area based on the gesture information and generating the boundary on the surface of the electronic whiteboard to obtain the selected area.

[0103] In one embodiment, the step of acquiring the identification object within the selected area and calling the corresponding operation menu on the electronic whiteboard surface according to the type of the identification object specifically includes: acquiring the identification object within the selected area; wherein the type of the identification object includes: image and text; when the type of the identification object is an image, a first floating menu is generated on the electronic whiteboard surface; wherein the operation options of the first floating menu include: image search, image deletion, image copy, and text selection; when the type of the identification object is text, a second floating menu is generated on the electronic whiteboard surface; wherein the operation options of the second floating menu include: font conversion, text search, text deletion, and text copy.

[0104] In one embodiment, when the type of the object to be identified is text, before generating the second floating menu on the electronic whiteboard surface, the method further includes: performing handwriting detection on the text within the selected area; when the text is handwritten, converting the handwritten text into standard text based on OCR technology; wherein the standard text is editable text.

[0105] In one embodiment, the instruction response module 203 is used to process the identified object in response to an operation instruction sent by the user on the operation menu, specifically including: when the operation instruction is an image search instruction, calling a first preset interface to perform an image search and displaying the image search results on the electronic whiteboard surface; when the operation instruction is an image deletion instruction, deleting the image within the selected area and re-rendering the electronic whiteboard surface; when the operation instruction is an image copy instruction, copying the image within the selected area to a preset position; when the operation instruction is a text selection instruction, extracting the text contained in the image within the selected area and copying it to the preset position.

[0106] In one embodiment, the instruction response module 203 is used to process the identified object in response to an operation instruction sent by the user on the operation menu, and further includes: when the operation instruction is a font conversion instruction, converting the text in the selected area into a preset font based on OCR technology and re-rendering the preset font on the electronic whiteboard surface; when the operation instruction is a text search instruction, calling a second preset interface to perform a text search and displaying the text search results on the electronic whiteboard surface; when the operation instruction is a text deletion instruction, deleting the text in the selected area and re-rendering the electronic whiteboard surface; when the operation instruction is a text copy instruction, copying the text in the selected area to a preset position.

[0107] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0108] This invention provides a whiteboard-based selection processing device that captures user touch events on the whiteboard surface. By recognizing these touch events, it generates the user-selected area, enabling real-time capture of the selection action and allowing the user to immediately see the selected area, thus reducing the probability of incorrect selection. The capture and recognition of touch events ensures the accuracy of the selected area, contributing to improved accuracy in subsequent operations. The device identifies the content within the selected area, determines the objects within the area, and generates corresponding operation menus based on the type of the identified objects. The operation options in the menus are related to the selected objects, making the operation process more intuitive and convenient. Users can complete operations with simple clicks, without needing to search for other menus or toolbars. By calling different operation menus, users can perform diverse operations on different types of content, meeting various usage scenarios. The instant display of the menus and the intuitive presentation of operation options allow users to better understand and use the functions of the whiteboard. Based on the user's instructions sent in the operation menu, the device processes the identified objects within the selected area accordingly, quickly responding to user operation commands and promptly processing objects, avoiding cumbersome operation steps. Furthermore, the operation menu is directly related to the selected object, which can reduce the risk of content modification or loss due to accidental operation.

[0109] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A selection processing method based on an electronic whiteboard, characterized in that, include: The system captures and identifies touch events on the surface of the electronic whiteboard, and generates a selected area based on the identification results. The selected area is used to identify objects, and the identified objects within the selected area are obtained. The corresponding operation menu is called on the surface of the electronic whiteboard according to the type of the identified objects. The operation menu includes several operation options. The identified object is processed in response to the operation command sent by the user on the operation menu.

2. The selection method based on an electronic whiteboard as described in claim 1, characterized in that, The process of capturing touch events on the electronic whiteboard surface, identifying those events, and generating a selected area based on the identification results specifically includes: The touch events on the electronic whiteboard surface are monitored in real time based on the touch screen driver; wherein the touch events include touch start events, touch movement events, and touch end events; Each time a touch event is detected, a touch point location information is generated. When the user stays on the electronic whiteboard surface for more than a first preset time, it is determined that the touch end event has been detected and the electronic whiteboard is controlled to enter the selection mode. The selected area is generated on the surface of the electronic whiteboard based on the accumulated first contact information.

3. The selection method based on an electronic whiteboard as described in claim 1, characterized in that, The process of capturing touch events on the surface of the electronic whiteboard, identifying the touch events, and generating a selected area based on the identification results also includes: Gesture information is obtained by capturing touch events on the surface of the electronic whiteboard based on a gesture recognition library; The boundary of the selected area is calculated based on the gesture information and the boundary is generated on the surface of the electronic whiteboard to obtain the selected area.

4. The selection method based on an electronic whiteboard as described in claim 1, characterized in that, The process of acquiring the identified object within the selected area and, based on the type of the identified object, calling the corresponding operation menu on the electronic whiteboard surface specifically includes: Obtain the objects to be identified within the selected area; wherein, the types of the objects to be identified include: images and text; When the type of the object to be identified is an image, a first floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the first floating menu include: image search, image deletion, image copy, and text selection; When the type of the identified object is text, a second floating menu is generated on the surface of the electronic whiteboard; wherein, the operation options of the second floating menu include: font conversion, text search, text deletion and text copy.

5. The selection method based on an electronic whiteboard as described in claim 4, characterized in that, When the type of the identified object is text, before generating the second floating menu on the electronic whiteboard surface, the following steps are also included: Handwriting detection is performed on the text within the selected area. When the text is handwritten, it is converted into standard text based on OCR technology; wherein the standard text is editable text.

6. The selection method based on an electronic whiteboard as described in claim 4, characterized in that, The object processing in response to the operation command sent by the user on the operation menu specifically includes: When the operation instruction is an image search instruction, the first preset interface is called to perform an image search and the image search results are displayed on the surface of the electronic whiteboard. When the operation instruction is an image deletion instruction, the image within the selected area is deleted and the electronic whiteboard surface is re-rendered; When the operation instruction is an image copy instruction, the image within the selected area is copied to a preset location; When the operation instruction is to select text, the text contained in the image within the selected area is extracted and copied to the preset location.

7. The selection method based on an electronic whiteboard as described in claim 4, characterized in that, The method of responding to an operation command sent by the user on the operation menu to perform object processing on the identified object further includes: When the operation instruction is a font conversion instruction, the text within the selected area is converted into a preset font based on OCR technology, and the preset font is re-rendered on the surface of the electronic whiteboard. When the operation instruction is a text search instruction, the second preset interface is invoked to perform a text search and the text search results are displayed on the surface of the electronic whiteboard. When the operation instruction is a text deletion instruction, the text within the selected area is deleted and the electronic whiteboard surface is re-rendered; When the operation instruction is a text copy instruction, the text within the selected area is copied to a preset position.

8. A selection processing device based on an electronic whiteboard, characterized in that, include: Touch recognition module, menu generation module, and command response module; The touch recognition module is used to capture touch events on the surface of the electronic whiteboard and recognize the touch events, and generate a selected area based on the recognition results; The menu generation module is used to perform object recognition on the selected area, obtain the recognized objects within the selected area, and call the corresponding operation menu on the electronic whiteboard surface according to the type of the recognized objects; wherein, the operation menu includes several operation options; The instruction response module is used to perform object processing on the identified object in response to the operation instructions sent by the user on the operation menu.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the whiteboard-based selection processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the selection processing method based on an electronic whiteboard as described in any one of claims 1 to 7.

Images