Electronic touch pen and input system thereof
By integrating image acquisition, pressure sensing and transmission modules in the electronic stylus, combining spatial gestures and touch, the problems of low parallelism and recognition accuracy in operation tasks in the prior art are solved, and more flexible and efficient interaction is achieved Experience.
Patent Information
- Application Number
- CN202211281031.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-19
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-10-19
AI Technical Summary
During the interaction process, the existing electronic stylus cannot be completed in parallel due to input conditions, which increases task completion time and hand fatigue. Due to different finger structures, the touch recognition accuracy is low, which affects the user experience.
An electronic stylus was designed, integrating an image acquisition module, a pressure sensing module and a transmission module. By acquiring gesture images and pressure information, generating operation information and adjustment information, adjusting the length and angle of the pen, realizing various input types Quickly input, combined with spatial gestures and touch, improve interactive flexibility.
It effectively improves the operation flexibility and recognition accuracy of the electronic stylus, expands the scope of application, reduces user operating time and hand fatigue, and improves user experience.
Smart Images

Figure CN115494971B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of electronic technology, and in particular relates to an electronic touch pen and an input system thereof. Background Art
[0002] At present, pen-based interaction and multi-touch interaction operated manually are intuitive and natural, and have always been considered to be promising interaction methods. Based on the traditional physical pen direct method, the electronic pen senses the pen pressure and pen holding posture, and other information input modes, thereby broadening the input bandwidth of the pen channel. In the information input link, fingers are directly used without other additional input media, so that users can directly obtain more tactile feedback during the interaction process. The inherent properties of touch interaction bring users more real experience. In daily human-computer interaction, users usually need to handle a variety of complex work tasks and switch between different task types to meet specific operation requirements. Due to the limitations of input conditions, operation tasks that could have been implemented in parallel are often forced to be completed in a sequential manner, which not only delays the task completion time, but also causes hand fatigue and affects the touch operation performance. Fat fingers and occlusion problems often cause misoperations in the process of directly using fingers to click or select small targets on touch devices, reducing the user experience. Summary of the invention
[0003] In view of this, the present invention provides an electronic stylus pen and an input system thereof that improves operational flexibility, recognition accuracy and expands the scope of application to solve the above-mentioned technical problems, which are specifically implemented by the following technical solutions.
[0004] In a first aspect, the present invention provides an electronic stylus pen for inputting information into an electronic device through a touch input module, the electronic stylus pen comprising:
[0005] Pen case;
[0006] A pen tip, disposed at the first end of the pen housing;
[0007] An image acquisition module, disposed at the second end of the pen housing, for acquiring a gesture image and generating operation information of the electronic stylus according to the gesture image, wherein the operation information includes a writing mode, a drawing mode and a control mode;
[0008] an adjustment module, connected to the image acquisition module, and configured to generate adjustment information according to the operation information and the gesture image, wherein the adjustment information includes the length of the electronic stylus and the angle of the image acquisition module;
[0009] a pressure sensing module, disposed in the pen tip, for obtaining a plurality of pressure information generated by the pen tip contacting the touch input module when the electronic stylus executes the operation information according to the adjustment information, wherein the plurality of pressure information respectively correspond to at least two contact areas of the electronic stylus contacting the touch input module;
[0010] The transmission module is arranged in the pen housing and is used together with the pressure sensing module to send multiple pressure information to the electronic device. The electronic device generates corresponding input information according to the trajectory information of the electronic stylus contacting the touch input module to execute the operation information and the multiple pressure information.
[0011] As a further improvement of the above technical solution, the pressure sensing module includes:
[0012] A setting unit, when the operation information is in writing mode, when the change value of the pressure information received by the pressure sensing module is less than a preset pressure value, the setting unit sets a first line corresponding to the pressure information; when the operation information is in drawing mode, when the change value of the pressure information is greater than or equal to the preset pressure, the setting unit sets a second line corresponding to the pressure information, and the width of the second line is greater than the width of the first line.
[0013] As a further improvement of the above technical solution, the pressure sensing module includes:
[0014] a conversion unit, connected to the setting unit, configured to convert the pressure information into a pressure value and send the pressure value to the setting unit;
[0015] The power supply unit is connected to the conversion unit and is used to supply power to each module in the pen housing.
[0016] As a further improvement of the above technical solution, the image acquisition module is used to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, including:
[0017] The number of fingers and the position of the palm in the collected gesture image are obtained. When the palm does not appear in the gesture image, the operation information is a writing mode; when the number of fingers is a single finger, the operation information is a control mode; when the number of fingers is multiple fingers, the operation information is a drawing mode.
[0018] As a further improvement of the above technical solution, the trajectory information includes the trajectory formed by the pen tip moving or clicking on the contact area on the touch input module, and the input information includes the input type and display color, and the input type includes at least one of text, picture or page.
[0019] In a second aspect, the present invention also provides an electronic stylus input system, comprising a wirelessly connected electronic stylus and an electronic device, wherein the electronic device comprises a gesture image preprocessing module, a touch input module and a control module, wherein the gesture image preprocessing module receives the gesture captured by the image acquisition module and sends it through a transmission module, and performs filtering processing to obtain a gesture image, wherein the control module generates operation information according to the gesture image, wherein the control module sends the operation information to the touch input module for display and adjustment of the electronic stylus, wherein the pressure sensing module receives the pressure information generated by the contact between the pen tip and the touch input module and sends it to the electronic device through the transmission module, and the control module generates corresponding input information according to the trajectory information and the pressure information.
[0020] As a further improvement of the above technical solution, the execution process of the gesture image preprocessing module includes:
[0021] The mean filtering algorithm is used to calculate the grayscale value of each point in the neighborhood of each pixel in the image. The image is preset as f(i,j), (i,j) is the pixel point on the image, and the expression corresponding to the mean filtering is Where M is the value range of image pixels covered by the calculation template, and n is the number of pixels contained in the neighborhood of point (i, j) covered by the calculation template operator.
[0022] As a further improvement of the above technical solution, when the image acquisition module detects a gesture, a Cartesian coordinate system is established, the origin of the coordinate system is at the center of the control module, the x-axis and the z-axis of the coordinate system are on the horizontal plane of the touch input module, the x-axis is parallel to the long side of the touch input module, the z-axis is parallel to the short side, and the y-axis of the coordinate system is perpendicular to the plane of the touch input module and the upward direction is the positive direction;
[0023] Each frame of image acquisition by the image acquisition module calculates and updates the data set according to the current image frame data, and each frame of data contains a data list. When the image acquisition module detects a gesture, the control module assigns a first identifier to the current gesture;
[0024] If the image acquisition module does not capture the gesture, the control module will assign a second identifier to the current gesture image, read the hand attributes according to the first identifier and the second identifier, and determine the gesture information.
[0025] As a further improvement of the above technical solution, gesture information includes static gestures and dynamic gestures. Static gestures use a combination of various finger joints and bending states of the hand to achieve gesture recognition, and dynamic gestures recognize different gestures through changes in the spatial position of the entire hand.
[0026] As a further improvement of the above technical solution, the hand attributes are read and the gesture information is determined according to the first identifier and the second identifier, including:
[0027] The three-dimensional gesture tracking evaluation algorithm is used to calculate the average Euclidean distance between the endpoints of the gesture in the gesture image and the true key points. The expression of the endpoint error is: The default single hand has 21 joints, i is the joint label number, D is the number of samples in the dataset, s is the sample number, w and h are the width and height of the original image, respectively, and y is the width and height of the original image. si represents the true value of the label, represents the predicted value;
[0028] The percentage of true key points is PCK, which is the percentage of true key points whose Euclidean distance error is below the threshold. Its expression is Where δ is the threshold. If the predicted value is less than the true value, the indicator function δ is set to 1, otherwise it is set to 0.
[0029] The present invention provides an electronic touch pen and an input system thereof. An image acquisition module, a pressure sensing module, an adjustment module and a transmission module are arranged in a stylus shell. When the electronic touch pen is used, the image acquisition module collects gesture information to determine operation information. The adjustment module adjusts the appropriate length of the electronic touch pen and the angle of the image acquisition module according to the operation information and the gesture, so as to facilitate the user to operate the electronic touch pen. The pressure sensing module can receive pressure information to determine the touch area and the thickness of the line. The transmission module transmits the pressure information and the trajectory information formed by the pen tip in the touch area to the electronic device and displays the corresponding input information on the touch input module. Combining spatial gestures with touch can realize fast input of various input types, and also improves the flexibility of interaction, effectively solving the problem of low touch recognition accuracy due to different hand structures, thereby improving the use range of the electronic touch pen and the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0031] Figure 1 A structural block diagram of the electronic touch pen provided by the present invention;
[0032] Figure 2 A structural block diagram of the electronic stylus input system provided by the present invention;
[0033] Figure 3 A structural block diagram of the pressure sensing module provided by the present invention;
[0034] Figure 4A diagram of the use process of the electronic touch pen provided by the present invention. DETAILED DESCRIPTION
[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0036] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0037] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0038] See also Figure 1 The present invention provides an electronic touch pen for inputting information into an electronic device through a touch input module, and the electronic touch pen comprises:
[0039] Pen case;
[0040] A pen tip, disposed at the first end of the pen housing;
[0041] An image acquisition module, disposed at the second end of the pen housing, for acquiring a gesture image and generating operation information of the electronic stylus according to the gesture image, wherein the operation information includes a writing mode, a drawing mode and a control mode;
[0042] an adjustment module, connected to the image acquisition module, and configured to generate adjustment information according to the operation information and the gesture image, wherein the adjustment information includes the length of the electronic stylus and the angle of the image acquisition module;
[0043] a pressure sensing module, disposed in the pen tip, for obtaining a plurality of pressure information generated by the pen tip contacting the touch input module when the electronic stylus executes the operation information according to the adjustment information, wherein the plurality of pressure information respectively correspond to at least two contact areas of the electronic stylus contacting the touch input module;
[0044] The transmission module is arranged in the pen housing and is used together with the pressure sensing module to send multiple pressure information to the electronic device. The electronic device generates corresponding input information according to the trajectory information of the electronic stylus contacting the touch input module to execute the operation information and the multiple pressure information.
[0045] In this embodiment, the pressure sensing module includes a setting unit, a conversion unit and a power supply unit. When the operation information is in writing mode, when the change value of the pressure information received by the pressure sensing module is less than the preset pressure value, the setting unit sets the first line corresponding to the pressure information; when the operation information is in drawing mode, when the change value of the pressure information is greater than or equal to the preset pressure, the setting unit sets the second line corresponding to the pressure information, and the width of the second line is greater than the width of the first line; the conversion unit is connected to the setting unit, and is used to convert the pressure information into a pressure value and send the pressure value to the setting unit; the power supply unit is connected to the conversion unit, and is used to supply power to each module in the pen shell. The pressure sensing module may include at least two pressure fiber bodies, and the pressure fiber bodies may be deformed according to the pressure received, and the degree of deformation of the pressure fiber bodies corresponds to the pressure. The pressure sensing module obtains at least two pressure information according to at least two pressure sensors. The image acquisition module may be installed at the top of the pen shell, and the pressure sensing module is set at one end close to the pen tip. The image acquisition module may collect user gestures to set the operation mode of the electronic stylus pen, which may be writing, drawing or controlling functions such as a mouse.
[0046] See also Figure 2It should be noted that the image acquisition module is a miniature camera, which is arranged at the end of the pen shell of the pen holder by means of structural inlay, and the two ends are connected to the two sides of the pen holder by a rotating shaft, and the rotating shaft is connected to the transmission motor through two meshing gears. The camera angle can be adjusted by controlling the motor display, and the electronic stylus can be appropriately adjusted according to the user's usage habits or operation mode. The transmission module can be a WeChat controller or a wireless communication function such as Bluetooth, which can display the pressure information received by the pressure sensing module and the trajectory formed by the pen tip in the touch input module to achieve fast and convenient interaction. The dynamic spatial gesture is realized in a waving manner. It is preset that the position data of the center of the palm is recorded once after a fixed number of frame data refreshes. The current dynamic gesture type is determined by the change of the palm position in the two frame data, and a dynamic gesture is recorded once. The change of the drawing color is realized by waving the palm left and right. Each dynamic gesture to the left or right will trigger a change in the brush color.
[0047] It should be understood that when the user uses an electronic stylus, the image acquisition module can recognize gestures and set operation information, so that the electronic stylus is suitable for different scenarios, and can be used for writing or drawing, or for simple control, and is easy to carry. The touch input module is a touch screen. Due to the physiological structure of the finger, the finger easily blocks the cursor, resulting in the inability to observe the relative position between the cursor and the baseline during the drawing process and finely adjust the fingertip position. The drawing offset is slightly larger than that under pen input, and the drawing time is slightly shorter than that under pen input. In the control mode, due to the friction between the finger and the screen, long-distance continuous sliding will cause discomfort on the finger contact surface, affecting the active experience of the operation. The combination of spatial gestures and touch makes the electronic stylus suitable for precise operations, which can avoid the hand discomfort and impact on the operating performance caused by long-distance touch sliding operations and long-term continuous spatial gesture operations.
[0048] Optionally, the image acquisition module is used to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, including:
[0049] The number of fingers and the position of the palm in the collected gesture image are obtained. When the palm does not appear in the gesture image, the operation information is a writing mode; when the number of fingers is a single finger, the operation information is a control mode; when the number of fingers is multiple fingers, the operation information is a drawing mode.
[0050] In this embodiment, the trajectory information includes the trajectory formed by the pen tip moving or clicking on the contact area on the touch input module, and the input information includes the input type and the display color, and the input type includes at least one of text, picture or page. The electronic stylus is a single-point input, and the input bandwidth is relatively high. The combination of multiple finger positions can express rich semantic information. The palm position can be used as a reference to determine the specific information of the gesture. In the related work of painting or design, most of them need to use an interactive digital screen or digital tablet for pen input, draw lines on the digital screen, click or select the color of the brush by color block, click to expand the folding menu and select the thickness of the brush, etc. Its operation tasks mainly include drawing tasks and command operation tasks. The user needs to move the input position of the pen and switch between the drawing area and the command operation area multiple times. The command operation that occurs during the drawing process will interrupt the current drawing task. When performing a drawing task with high accuracy requirements, after the user's drawing process is interrupted by the command operation, it is necessary to accurately adjust the position of the hand and pen multiple times before returning to the drawing state, which causes a serious waste of time and reduces the input efficiency. If multiple command operations are required during the drawing process, such as changing the color of the stroke, the adjustment process before each switch back to the drawing task will accumulate, causing fatigue in the upper arm and hand, and causing psychological fatigue to the user, thereby affecting the ability to perform drawing and command operations.
[0051] It should be noted that when the operation information is in writing mode, the appropriate length is adjusted according to the user's hand structure. When the operation information is in drawing mode, the camera angle can be adjusted to quickly capture gesture images, thereby realizing interface switching or color selection, etc. When the operation information is in control mode, the length of the electronic stylus can also be adjusted to facilitate clicking or sliding to replace the function of the mouse. Writing can replace the simple input function of the keyboard. In actual use, the user can choose to operate the electronic stylus with one hand or with both hands, and distribute the operating hand according to the accuracy of the operation and cognitive load. Cognitive load is related to working memory, and the brain needs to mobilize resources for short-term memory storage. Information processing and action control have completed the corresponding tasks. Under touch operation without restricted area, if you choose to combine touch and pen operation on the right hand, the operation of the touch command gesture causes the pen tip to move, and the user may readjust the pen holding posture in order to perform touch operation with the right hand, which improves the convenience of using the electronic stylus.
[0052] See also Figure 2The present invention also provides an electronic stylus input system, comprising an electronic stylus and an electronic device that are wirelessly connected, wherein the electronic device comprises a gesture image preprocessing module, a touch input module and a control module, wherein the gesture image preprocessing module receives the gesture collected by the image acquisition module and sends it to the transmission module, and performs filtering processing to obtain a gesture image, wherein the control module generates operation information according to the gesture image, wherein the control module sends the operation information to the touch input module for display and adjustment of the electronic stylus, wherein the pressure sensing module receives the pressure information generated by the contact between the pen tip and the touch input module and sends it to the electronic device through the transmission module, and the control module generates corresponding input information according to the trajectory information and the pressure information.
[0053] In this embodiment, the execution process of the gesture image preprocessing module includes: using the mean filtering algorithm to calculate the grayscale value of each point in the neighborhood of each pixel point of the image, presetting an image as f(i, j), (i, j) is the pixel point on the image, and the expression corresponding to the mean filtering is: Where M is the value range of image pixels covered by the calculation template, and n is the number of pixels in the neighborhood of point (i, j) covered by the calculation template operator. The use process of the electronic stylus includes the following steps:
[0054] S1: collecting a gesture image and generating operation information of the electronic stylus according to the gesture image, wherein the operation information includes a writing mode, a drawing mode and a control mode;
[0055] S2: Generate adjustment information according to the operation information and the gesture image, wherein the adjustment information includes the length of the electronic stylus and the angle of the image acquisition module;
[0056] S3: when the electronic stylus executes the operation information according to the adjustment information, a plurality of pressure information generated by the pen tip contacting the touch input module is obtained, where the plurality of pressure information respectively correspond to at least two contact areas where the electronic stylus contacts the touch input module;
[0057] S4: sending a plurality of pressure information to the electronic device, and the electronic device generates corresponding input information according to the trajectory information of the electronic stylus contacting the touch input module to execute the operation information and the plurality of pressure information.
[0058] It should be noted that for users whose dominant input hand is the right hand, the operating hand for pen input is the right hand, while touch and spatial gestures can be operated with the left or right hand. When using the electronic stylus as the main input channel, it is necessary to ensure that the hand holding the pen is the user's right hand. On the basis of using the right hand to hold the pen to achieve precise input, the left hand can operate touch or spatial gestures to achieve mixed input, including two allocation methods: left-hand touch, right-hand pen and left-hand spatial gestures, right-hand pen. The electronic stylus can be connected to the electronic device via Bluetooth or other wireless modes. The electronic stylus is also equipped with operation buttons to better replace the mouse function. The operating hand can also be allocated according to the distance the pen tip moves. When the electronic device can clearly distinguish between touch and pen input behaviors, the position of the touch operation can be unlimited, and touch gestures can be used to achieve better input. When the touch and pen are mixed for input, the touch is used to control the input properties of the pen. The area of the operation space is the operation surface of the interactive digital screen, that is, a two-dimensional plane of fixed size. The current operation plane is divided to expand the number of operable input properties. The electronic device can store the interactive result of the user's first writing or control, and then push the interactive result the next time the input is made. The user can also use the electronic stylus to switch pages or play quickly on the electronic device, thereby expanding the application scope of the electronic stylus.
[0059] Optionally, when the image acquisition module detects a gesture, a Cartesian coordinate system is established, the origin of the coordinate system is at the center of the control module, the x-axis and z-axis of the coordinate system are on the horizontal plane of the touch input module, the x-axis is parallel to the long side of the touch input module, the z-axis is parallel to the short side, and the y-axis of the coordinate system is perpendicular to the plane of the touch input module and the positive direction is upward;
[0060] Each frame of image acquisition by the image acquisition module calculates and updates the data set according to the current image frame data, and each frame of data contains a data list. When the image acquisition module detects a gesture, the control module assigns a first identifier to the current gesture;
[0061] If the image acquisition module does not capture the gesture, the control module will assign a second identifier to the current gesture image, read the hand attributes according to the first identifier and the second identifier, and determine the gesture information.
[0062] In this embodiment, gesture information includes static gestures and dynamic gestures. Static gestures use a combination of finger joints and bending states of the hand to achieve gesture recognition, and dynamic gestures recognize different gestures by the change in the spatial position of the entire hand. Reading the hand attributes according to the first identifier and the second identifier and determining gesture information includes: using a three-dimensional gesture tracking evaluation algorithm to calculate the average Euclidean distance between the endpoints of the gesture and the real key points in the gesture image, and the expression of the endpoint error is: The default single hand has 21 joints, i is the joint label number, D is the number of samples in the dataset, s is the sample number, w and h are the width and height of the original image, respectively, and y is the width and height of the original image. si represents the true value of the label, represents the predicted value; the percentage of true key points is PCK, that is, the percentage of true key points whose Euclidean distance error is lower than the threshold, and its expression is Where δ is the threshold. If the predicted value is less than the true value, the indicator function δ is set to 1, otherwise it is set to 0.
[0063] In another feasible embodiment, the electronic stylus also includes a built-in microprocessor, a wireless module and a storage unit. The microprocessor mainly completes the clock and instruction control of the flash memory, the storage unit is used to realize the storage of all static information and dynamic process power-off files, and the wireless module is used to realize the real-time wireless transmission function connected to the electronic device. The storage process of the electronic stylus is as follows: first, the information of the electronic device is received through the data line or the wireless carrier signal is received through the wireless antenna to the wireless module for adjustment, and then the information is transmitted to the memory card interface control chip to control the normal reading and writing of the clock and data. The control chip stores the information file in the storage unit of the electronic stylus through the storage interface. The stylus transmission process is as follows: insert the stylus to directly connect the electronic device, and use moving or copying and pasting to transfer information between the stylus and the electronic device; or, control the stylus to send or receive data by operating the stylus on the touch input module of the electronic device, i.e., the screen. When the stylus completes the pen gesture operation on the screen, the control module of the electronic device executes an instruction to determine whether to send or receive. When the stylus sends, it first needs to receive the control instruction from the electronic device, and then sends the information document required by the user in the stylus to the electronic device; when receiving, the user can select to transfer the edited text, picture or application process to the stylus.
[0064] The present invention provides an electronic touch pen input system, which arranges an image acquisition module, a pressure sensing module, an adjustment module and a transmission module in a stylus shell. When the electronic touch pen is used, the image acquisition module collects gesture information to determine operation information. The adjustment module adjusts the appropriate length of the electronic touch pen and the angle of the image acquisition module according to the operation information and the gesture, so as to facilitate the user to operate the electronic touch pen. The pressure sensing module can receive pressure information to determine the touch area and line thickness. The transmission module transmits the pressure information and the trajectory information formed by the pen tip in the touch area to the electronic device and displays the corresponding input information on the touch input module. Combining spatial gestures with touch can realize fast input of multiple input types, and also improves the flexibility of interaction, effectively solving the problem of low touch recognition accuracy due to different hand structures, thereby improving the use range of the electronic touch pen and the user experience.
[0065] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.
[0066] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0067] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.
Claims
1. An electronic stylus pen, used to input information into an electronic device through a touch input module, characterized in that: The electronic stylus comprises: Pen case; A pen tip, disposed at the first end of the pen housing; An image acquisition module, disposed at the second end of the pen housing, for acquiring a gesture image and generating operation information of the electronic stylus according to the gesture image, wherein the operation information includes a writing mode, a drawing mode and a control mode; an adjustment module, connected to the image acquisition module, and used to generate adjustment information according to the operation information and the gesture image, wherein the adjustment information is used to adjust the length of the electronic stylus and the angle of the image acquisition module; a pressure sensing module, disposed in the pen tip, for obtaining a plurality of pressure information generated by the pen tip contacting the touch input module when the electronic stylus executes the operation information according to the adjustment information, wherein the plurality of pressure information respectively correspond to at least two contact areas of the electronic stylus contacting the touch input module; A transmission module, disposed in the pen housing, and the pressure sensing module, for sending a plurality of pressure information to the electronic device, and the electronic device generates corresponding input information according to the trajectory information of the electronic stylus contacting the touch input module to execute the operation information and the plurality of pressure information; a setting unit, wherein when the operation information is in writing mode, when the change value of the pressure information received by the pressure sensing module is less than a preset pressure value, the setting unit sets a first line corresponding to the pressure information; when the operation information is in drawing mode, when the change value of the pressure information is greater than or equal to the preset pressure, the setting unit sets a second line corresponding to the pressure information, and the width of the second line is greater than the width of the first line; the pressure sensing module comprises: a conversion unit, connected to the setting unit, for converting the pressure information into a pressure value, and sending the pressure value to the setting unit; A power supply unit, connected to the conversion unit, for supplying power to each module in the pen housing; The trajectory information includes the trajectory formed by the movement or clicking of the pen tip on the contact area on the touch input module, and the input information includes the input type and display color, and the input type includes at least one of text, picture or page; the image acquisition module is a miniature camera, which is arranged at the end of the pen shell of the pen holder through a structural inlay method, and the two ends are connected to the two sides of the pen holder through a rotating shaft, and the rotating shaft is connected to the transmission motor through two meshing gears, and the camera angle is adjusted by controlling the motor display.
2. The electronic stylus pen according to claim 1, characterized in that: The image acquisition module is used to acquire a gesture image and generate operation information of the electronic stylus according to the gesture image, including: The number of fingers and the position of the palm in the collected gesture image are obtained. When the palm does not appear in the gesture image, the operation information is a writing mode; when the number of fingers is a single finger, the operation information is a control mode; when the number of fingers is multiple fingers, the operation information is a drawing mode.
3. An electronic stylus input system according to any one of claims 1 to 2, characterized in that: It includes a wirelessly connected electronic stylus and an electronic device, the electronic device including a gesture image preprocessing module, a touch input module and a control module, the gesture image preprocessing module receives the gesture collected by the image acquisition module and sends it to the transmission module, and performs filtering processing to obtain a gesture image, the control module generates operation information according to the gesture image, the control module sends the operation information to the touch input module for display and adjustment of the electronic stylus, the pressure sensing module receives the pressure information generated by the contact between the pen tip and the touch input module and sends it to the electronic device through the transmission module, and the control module generates corresponding input information according to the trajectory information and the pressure information.
4. The electronic stylus input system according to claim 3, characterized in that: The execution process of the gesture image preprocessing module includes: The mean filter algorithm is used to calculate the grayscale value of each point in the neighborhood of each pixel in the image. The image is preset as f(i,j), (i,j) is the pixel point on the image, and the expression corresponding to the mean filter is: Where M is the value range of image pixels covered by the calculation template, and n is the number of pixels contained in the neighborhood of point (i, j) covered by the calculation template operator.
5. The electronic stylus input system according to claim 3, characterized in that: When the image acquisition module detects a gesture, a Cartesian coordinate system is established, with the origin of the coordinate system at the center of the control module, the x-axis and z-axis of the coordinate system on the horizontal plane of the touch input module, the x-axis is parallel to the long side of the touch input module, the z-axis is parallel to the short side, and the y-axis of the coordinate system is perpendicular to the plane of the touch input module and the upward direction is the positive direction; The image acquisition module calculates each frame of the acquired image to obtain the current image frame data and updates the data set corresponding to the image frame data. Each frame of data contains a data list. When the image acquisition module detects a gesture, the control module assigns a first identifier to the current gesture. If the image acquisition module does not capture the gesture, the control module will assign a second identifier to the current gesture image, read the hand attributes according to the first identifier and the second identifier, and determine the gesture information.
6. The electronic stylus input system according to claim 5, characterized in that: Gesture information includes static gestures and dynamic gestures. Static gestures use a combination of the hand's finger joints and bending states to achieve gesture recognition, while dynamic gestures recognize different gestures through changes in the spatial position of the entire hand.
7. The electronic stylus input system according to claim 5, characterized in that: Reading hand attributes according to the first identifier and the second identifier and determining gesture information includes: The three-dimensional gesture tracking evaluation algorithm is used to calculate the average Euclidean distance between the endpoints of the gesture in the gesture image and the true key points. The expression of the endpoint error is: The default single hand has 21 joints, i is the joint label number, D is the number of samples in the dataset, s is the sample number, w and h are the width and height of the original image, respectively, and y is the width and height of the original image. si represents the true value of the label, represents the predicted value; The percentage of true key points is PCK, which is the percentage of true key points whose Euclidean distance error is below the threshold. Its expression is Where δ is the threshold. If the predicted value is less than the true value, the indicator function δ is set to 1, otherwise it is set to 0.
