Human-computer interaction method of smart pen holder, smart pen holder and storage medium

Abstract

The present application belongs to the field of image processing technology, and more particularly relates to a human-computer interaction method of an intelligent pen holder, an intelligent pen holder and a storage medium, wherein the method can monitor the remaining amount of ink in a pen used by a user in real time during the user's writing task, and promptly determine a pen with similar characteristics to the pen being used from the pen holder when the ink is running out, thereby enabling the user to quickly replace the pen when the ink is used up, avoiding time waste when replacing the pen and improving the user's writing efficiency.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a human-computer interaction method of a smart pen holder, a smart pen holder and a storage medium. Background Art

[0002] The smart pen holder is an innovative product that combines traditional stationery functions with modern smart technology. It can not only be used as an ordinary pen holder to store stationery, but also has a variety of intelligent interactive functions, such as information push, schedule reminders, etc.

[0003] However, the shells of most pens are currently non-transparent, making it difficult for users to easily determine the remaining ink in the pen they are using. When the ink in the pen they are using is used up, the smart pen holder cannot help them determine the replaceable pens in the pen holder in time. Users can only try out the pens in the pen holder one by one, which wastes a lot of time. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a human-computer interaction method of a smart pen holder, a smart pen holder and a storage medium, which can solve the above technical problems.

[0005] A first aspect of an embodiment of the present application provides a human-computer interaction method for a smart pen holder, which is applied to a smart pen holder. The human-computer interaction method for the smart pen holder includes:

[0006] S1: when receiving a task type of a writing task input by a user, determining a first pen that matches the task type;

[0007] S2: capturing a top view of the smart pen holder through the auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder;

[0008] S3: after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on the display screen;

[0009] S4: When the user performs a writing task, the auxiliary camera is used to monitor a movement trajectory of the tip of the first pen on the paper, and the length of the movement trajectory is identified;

[0010] S5: determining a unit ink consumption percentage of the first pen, calculating a real-time ink consumption percentage of the first pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time, wherein the unit ink consumption percentage is the ink consumption percentage after the tip of the first pen moves on the paper by a unit track length;

[0011] S6: when the remaining ink percentage is lower than a preset value, inquiring the user about the remaining writing amount percentage, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount percentage, and if so, suggesting the user to continue using the first pen;

[0012] S7: If not, a second pen for replacing the first pen is determined from other pens in the pen holder according to the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task;

[0013] S8: Repeat steps S4 to S7 until the user completes the writing task.

[0014] A second aspect of an embodiment of the present application provides a smart pen holder, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method of the smart pen holder.

[0015] A third aspect of an embodiment of the present application provides a terminal-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the human-computer interaction method of the smart pen holder.

[0016] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the method provided by the present invention includes, when receiving a task type of a writing task input by a user, determining a first pen that matches the task type; capturing a top view of the smart pen holder through an auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder; after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on a display screen; when the user performs a writing task, monitoring the movement trajectory of the pen tip of the first pen on the paper through the auxiliary camera, and identifying the trajectory length of the movement trajectory; determining the unit ink consumption percentage of the first pen, calculating the real-time ink consumption percentage of the first pen based on the unit ink consumption percentage of the first pen and the trajectory length, and then updating the remaining ink percentage in real time; when the remaining ink percentage is lower than a preset value, asking The user is queried about the remaining writing volume ratio, and based on the remaining writing volume ratio, it is determined whether the user can complete the writing task with the remaining ink of the first pen, if so, it is recommended that the user continue to use the first pen; if not, a second pen for replacing the first pen is determined from other pens in the pen holder based on the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task; the above steps are repeated until the user completes the writing task; in the present application, in the process of the user performing the writing task, the remaining ink volume of the pen used by the user can be monitored in real time, and when the ink is running out, a pen with similar characteristics to the pen currently being used can be promptly determined from the pen holder, so that the user can quickly complete the replacement of the pen when the ink is used up, avoiding time waste when replacing the pen, and improving the user's writing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 It is a schematic diagram of the implementation flow of the human-computer interaction method of the smart pen holder provided in the embodiment of the present application;

[0019] Figure 2 Schematic diagram of the implementation environment of the human-computer interaction method of the smart pen holder provided in the embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a target area of ​​a human-computer interaction method of a smart pen holder provided in an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the smart pen holder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0023] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0024] Figure 1 A human-computer interaction method of a smart pen holder provided in the first embodiment of the present application is shown, which is applied to the smart pen holder. The human-computer interaction method of the smart pen holder includes:

[0025] S1: when receiving a task type of a writing task input by a user, determining a first pen that matches the task type;

[0026] S2: capturing a top view of the smart pen holder through the auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder;

[0027] S3: after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on the display screen;

[0028] S4: When the user performs a writing task, the auxiliary camera is used to monitor a movement trajectory of the tip of the first pen on the paper, and the length of the movement trajectory is identified;

[0029] S5: determining a unit ink consumption percentage of the first pen, calculating a real-time ink consumption percentage of the first pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time, wherein the unit ink consumption percentage is the ink consumption percentage after the tip of the first pen moves on the paper by a unit track length;

[0030] S6: when the remaining ink percentage is lower than a preset value, inquiring the user about the remaining writing amount percentage, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount percentage, and if so, suggesting the user to continue using the first pen;

[0031] S7: If not, a second pen for replacing the first pen is determined from other pens in the pen holder according to the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task;

[0032] S8: Repeat steps S4 to S7 until the user completes the writing task.

[0033] In this embodiment, if Figure 2 As shown, the method is executed in a smart pen holder, which is provided with a display screen, and then interactive information is displayed on the display screen to perform human-computer interaction with the user; the smart pen holder is often placed on a desk, and an auxiliary camera (a camera with a rotatable viewing angle) is provided in conjunction with the smart pen holder. The auxiliary camera is arranged above the smart pen holder, and can capture a top view of the smart pen holder, and can also capture a video image of the user writing on the desk. The smart pen holder communicates with the auxiliary camera, and then can obtain the image captured by the camera.

[0034] In this embodiment, each user needs to register the user through the display screen on the smart pen holder when using the method for the first time. During the registration, the display screen will display an authorization letter, which includes an introduction to the operation steps of the method and the corresponding information to be collected. After the user agrees to the authorization letter and completes the registration, the smart pen holder can execute the method for the user;

[0035] In this embodiment, each time a user uses the smart pen holder, the user first logs in through the display screen; then enters the task type on the display screen, such as design sketches, research reports, etc. Each user can set a mapping table of task types and matching pens in the smart pen holder in advance (map each task type with the model and appearance image of the matching pen), and each time the smart pen holder receives the task type input by the user, it can immediately determine the corresponding first pen; for each pen placed in the pen holder, the smart pen holder can identify its appearance characteristics through the auxiliary camera, and monitor its placement and movement position, and then determine the position of each model in the top view in real time; the mark made for the first pen can be a red circle, or other marks, which are not limited here; after seeing the mark, the user can easily find the first pen and take it out;

[0036] In this embodiment, since this method can update the remaining ink percentage in real time, after each use of the pen by the user, the smart pen holder can identify and record the latest remaining ink percentage, and the next time the user uses the pen, the latest remaining ink percentage can be called out as the initial remaining ink percentage;

[0037] In this embodiment, since the ink consumption is proportional to the length of the moving track of the pen tip (the longer the length, the more ink is produced), this embodiment can estimate the ink consumption in real time by monitoring the moving length of the pen tip, and then calculate the real-time remaining ink amount, so that the user can determine the remaining ink without disassembling the pen case to take out the refill; the track length identified in this embodiment is only the track length when the pen tip is producing ink, and if the pen tip is not producing ink, it will not be identified;

[0038] In this embodiment, there is a task completion button on the display screen. When the smart pen holder recognizes that the user has clicked the button, it indicates that the user has completed the writing task. At this time, the smart pen holder can generate instruction information, instructing the user to put the pen that has not run out of ink back into the pen holder, and update the pen that has run out of ink (replace the pen or replace the refill) and then put it back into the pen holder.

[0039] In the present application, while the user is performing a writing task, the remaining amount of ink in the pen used by the user can be monitored in real time, and when the ink is running out, a pen with similar characteristics to the pen being used can be promptly determined from the pen holder. This allows the user to quickly replace the pen when the ink runs out, avoiding wasting time when replacing the pen and improving the user's writing efficiency.

[0040] As a preferred embodiment, determining the unit ink consumption percentage of the first pen, calculating the real-time ink consumption percentage of the habitual pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time includes:

[0041] Obtaining the pen records of all historical pens of the same model as the first pen, wherein the pen records include the usage time interval of the corresponding historical pen and the total length of the movement track corresponding to the historical pen in the usage time interval;

[0042] For each historical pen, divide 100% by the total length to get the ink consumption efficiency of the historical pen;

[0043] Calculate the average of the ink consumption efficiency of each historical pen, and obtain the unit ink consumption percentage of the first pen;

[0044] Each time the length of the newly added track of the first pen is detected, the real-time ink consumption percentage is obtained by multiplying the track length by the unit ink consumption percentage;

[0045] Subtract the real-time ink consumption percentage from the latest remaining ink percentage to obtain the updated remaining ink percentage.

[0046] In this embodiment, the monitoring data of each pen of the smart pen holder can be stored in the cloud, and historical data can be called out from the cloud when needed, such as the historical pen usage records; the usage time interval of the historical pen, that is, the time point when the tip of the historical pen moves on the paper for the first time to the time point when the ink of the historical pen is used up (the pen tip continues to not produce ink on the paper, and the user checks to confirm that there is no remaining ink, and the user clicks on the screen that the ink is exhausted), the comprehensive length of all the trajectory lengths of the historical pen monitored in the usage time interval, that is, the total length of the moving trajectory; the ink consumption efficiency, that is, the percentage of ink consumption corresponding to the unit length of the trajectory; in this embodiment, the obtained unit ink consumption percentage is the average of the historical values, which has higher accuracy, thereby making the prediction of the remaining ink percentage of the first pen have higher accuracy.

[0047] As a preferred embodiment, the preset value is a preset percentage; inquiring the user about the remaining writing amount ratio, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount ratio includes:

[0048] Inquiring the user about the remaining writing volume ratio, so that the user can input the remaining writing volume ratio on the screen;

[0049] Subtract the remaining writing percentage from 100% to get the completed writing percentage.

[0050] Retrieve the track length corresponding to the completed writing amount, and then divide the retrieved track length by the proportion of the completed writing amount to obtain the total track length;

[0051] Calculating a first remaining track length corresponding to the writing task according to the total track length and the remaining writing amount ratio;

[0052] Calculating the second remaining track length corresponding to the first pen stroke according to the preset value and the unit ink consumption percentage;

[0053] The first remaining track length and the second remaining track length are compared. If the first track length is greater than the second track length, the user cannot complete the writing task with the remaining ink of the first pen. Otherwise, the user can complete the writing task with the remaining ink of the first pen.

[0054] The first remaining trajectory length is calculated by the following formula:

[0055]

[0056] in, is the first remaining trajectory length, is the total trajectory length of the writing task, is the proportion of remaining writing volume;

[0057] The second remaining trajectory length is calculated by the following formula:

[0058]

[0059] in, is the second remaining trajectory length, is the default value, It is the percentage of unit ink consumption.

[0060] In this embodiment, the preset value may be 20%, or other values; by setting the preset value, it is possible to determine in advance whether the user needs to change pens, and if so, which pen to change to, which can further improve the user's pen-changing efficiency; furthermore, since the user's estimate of the remaining writing volume may be biased, after receiving the remaining writing volume ratio (the ratio of the remaining writing task volume to the total writing task volume) input by the user, the present application will multiply the input value by a coefficient greater than 1, so that the first remaining track length calculated subsequently is longer, thereby avoiding the situation where the user runs out of ink and has not determined to replace the pen due to the estimation deviation; completed The writing volume ratio refers to the ratio of the task volume of the completed part of the writing task to the total task volume; the trajectory length corresponding to the completed writing volume is retrieved, that is, the sum of the trajectory lengths of the first pen monitored from the beginning of this writing task to the current one; the total trajectory length is the predicted length that the tip of the first pen needs to move on the paper to complete this writing task; the first remaining trajectory length represents the length of the first pen required to move for the remaining task volume, and the second remaining trajectory length represents the length that the remaining ink of the first pen can maintain its movement. By comparing the first remaining trajectory length and the second remaining trajectory length, it can be intuitively and clearly determined whether the user can rely on the remaining ink of the first pen to complete the writing task.

[0061] As a preferred embodiment, determining the second pen for replacing the first pen from other pens in the pen holder according to the characteristics of the first pen comprises:

[0062] S71: Determine writing characteristics of a first pen, wherein the writing characteristics include ink color and pen tip size;

[0063] S72: Determine, from other pens in the pen holder, a reserve pen whose writing characteristics have a deviation from the writing characteristics of the first pen less than a preset deviation;

[0064] S73: determining the pen holding gesture characteristics of the user, and then determining the corresponding pen holding comfort according to the shape and pen holding gesture characteristics of each candidate pen;

[0065] S74: Determine the alternative pen with the highest corresponding pen holding comfort as the second pen.

[0066] The preset deviation includes a preset color deviation corresponding to the ink color and a preset diameter deviation corresponding to the pen tip diameter; the alternative pens determined from other pens in the pen holder whose writing characteristics have a deviation less than the preset deviation from the writing characteristics of the first pen include:

[0067] S721: Select a pen from other pens in the pen holder as a comparison pen;

[0068] S722: Determine whether the chromaticity deviation between the ink color of the comparison pen and the first pen is less than a preset chromaticity deviation, if not, the comparison pen is not a reserve pen;

[0069] S723: If yes, determine whether the deviation between the tip diameters of the comparison pen and the first pen is less than a preset diameter deviation; if no, the comparison pen is not a reserve pen; if yes, the comparison pen is a reserve pen;

[0070] S724: Repeat steps S721 to S723 until all alternative pens are determined.

[0071] like Figure 3 As shown, determining the pen holding gesture characteristics of the user, and then determining the corresponding pen holding comfort according to the shape and pen holding gesture characteristics of each candidate pen includes:

[0072] For each candidate pen, generate a three-dimensional pen model of the candidate pen;

[0073] Capture the user's pen-holding gesture image through the auxiliary camera;

[0074] Generate a three-dimensional model of the user's hand when holding the pen based on the gesture image;

[0075] Retrieving the user's historical gesture images, determining the pen holding distance between the pen holding point and the pen tip in each historical gesture image, and calculating the average of the pen holding distances, wherein the pen holding point is the midpoint of the pen cross section where the thumb and index finger are located;

[0076] Identify the support area on each historical gesture image, and generate a marker area at the corresponding position of the three-dimensional hand model, wherein the support area is the area of ​​the base of the user's hand used to support the pen holder;

[0077] Identify the target area on the hand 3D model where the support areas with a set number of layers overlap, retain the target area, and remove other support areas;

[0078] Placing the 3D pen model on the 3D hand model, adjusting the position of the 3D pen model so that the holding distance of the 3D hand model to the holding point of the 3D pen model is equal to the average holding distance, and the pen holder of the 3D pen model is against the 3D hand model;

[0079] The contact area between the three-dimensional hand model and the three-dimensional pen model is identified, and the overlap rate between the contact area and the target area is determined, and then the holding comfort of the alternative pen is determined according to the overlap rate.

[0080] In this embodiment, The value can be 10 or other values; the smart pen holder has built-in AI 3D modeling software (such as Sudo AI), which can extract features of the objects in the captured image (such as the user's hand, pen), and then generate a corresponding model based on the extracted features; the first pen feature includes writing features and pen holding gesture features; writing features include pen ink color and pen tip size; pen holding gesture features include pen holding distance and support area; the preset chromaticity deviation can be 10 (or other preset values), that is, the deviation of the pen ink color in the three chromaticity channels of red, green and blue is less than 10; the preset diameter deviation can be 0.05mm or other values; the historical gesture image is the gesture image of the user when holding the pen captured by the auxiliary camera in the past. Each user has a specific pen holding habit, so the gestures in each historical gesture image are roughly the same; furthermore, the appearance of a pen The more the shape conforms to the user's writing habit, the higher the user's comfort level will be; the writing habit is mainly reflected in the writing distance and the support area. The present embodiment can simulate the overlap between the support area of ​​the alternative pen and the user's comfort area while ensuring the user's writing distance. The higher the overlap, the higher the comfort level can be represented. The comfort area is the target area. Since the target area is an area where multiple layers (a set number of layers, such as 10 layers) of support areas overlap, it can represent the comfort area of ​​the user's holding pen. In the present embodiment, not only is the determined second pen similar to the first pen in function, thereby ensuring the consistency of the font color and size before and after the writing task, but it can also fully adapt to the user's writing habit, thereby ensuring the user's comfort level after changing the pen.

[0081] A smart pen holder provided in the second embodiment of the present application includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method of the smart pen holder, specifically including:

[0082] S1: when receiving a task type of a writing task input by a user, determining a first pen that matches the task type;

[0083] S2: capturing a top view of the smart pen holder through the auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder;

[0084] S3: after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on the display screen;

[0085] S4: When the user performs a writing task, the auxiliary camera is used to monitor a movement trajectory of the tip of the first pen on the paper, and the length of the movement trajectory is identified;

[0086] S5: determining a unit ink consumption percentage of the first pen, calculating a real-time ink consumption percentage of the first pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time, wherein the unit ink consumption percentage is the ink consumption percentage after the tip of the first pen moves on the paper by a unit track length;

[0087] S6: when the remaining ink percentage is lower than a preset value, inquiring the user about the remaining writing amount percentage, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount percentage, and if so, suggesting the user to continue using the first pen;

[0088] S7: If not, a second pen for replacing the first pen is determined from other pens in the pen holder according to the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task;

[0089] S8: Repeat steps S4 to S7 until the user completes the writing task.

[0090] A terminal-readable storage medium is provided in the third embodiment of the present application. A computer program is stored on the terminal-readable storage medium. When the computer program is executed by a processor, the processor executes the steps of the human-computer interaction method of the smart pen holder, specifically including:

[0091] S1: when receiving a task type of a writing task input by a user, determining a first pen that matches the task type;

[0092] S2: capturing a top view of the smart pen holder through the auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder;

[0093] S3: after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on the display screen;

[0094] S4: When the user performs a writing task, the auxiliary camera is used to monitor a movement trajectory of the tip of the first pen on the paper, and the length of the movement trajectory is identified;

[0095] S5: determining a unit ink consumption percentage of the first pen, calculating a real-time ink consumption percentage of the first pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time, wherein the unit ink consumption percentage is the ink consumption percentage after the tip of the first pen moves on the paper by a unit track length;

[0096] S6: when the remaining ink percentage is lower than a preset value, inquiring the user about the remaining writing amount percentage, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount percentage, and if so, suggesting the user to continue using the first pen;

[0097] S7: If not, a second pen for replacing the first pen is determined from other pens in the pen holder according to the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task;

[0098] S8: Repeat steps S4 to S7 until the user completes the writing task.

[0099] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0100] It should be understood that when used in the present application specification, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0101] It should also be understood that the term “and / or” used in the specification of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0102] As used in the present specification, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0103] In addition, in the description of the present specification, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used to describe various elements in some embodiments of the present application in the text, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first table can be named as the second table, and similarly, the second table can be named as the first table, without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0104] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0105] Figure 4 Schematic diagram of the structure of the smart pen holder provided by an embodiment of the present application. Figure 4 As shown, the smart pen holder of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the figure), a memory, wherein the memory stores a computer program that can be run on the processor. When the processor executes the computer program, the steps in the above-mentioned human-computer interaction method embodiments of the smart pen holder are implemented, such as Figure 1 Steps S1 to S8 are shown.

[0106] The smart pen holder can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The smart pen holder can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 4 It is only an example of a smart pen holder and does not constitute a limitation of the smart pen holder. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smart pen holder may also include an input sending device, a network access device, a bus, etc.

[0107] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0108] In some embodiments, the memory may be an internal storage unit of the smart pen holder, such as a hard disk or memory of the smart pen holder. The memory may also be an external storage device of the smart pen holder, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart pen holder. Furthermore, the memory may include both an internal storage unit and an external storage device of the smart pen holder. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory may also be used to temporarily store data that has been sent or is to be sent.

[0109] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0110] An embodiment of the present application also provides a smart pen holder, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the smart pen holder implements the steps in any of the above-mentioned method embodiments.

[0111] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0112] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile smart pen holder, the mobile smart pen holder can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0113] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A human-computer interaction method for a smart pen holder, applied to the smart pen holder, characterized in that: The human-computer interaction method of the smart pen holder comprises: S1: when receiving a task type of a writing task input by a user, determining a first pen that matches the task type; S2: capturing a top view of the smart pen holder through the auxiliary camera, and marking the first pen in the top view to guide the user to take the first pen out of the pen holder; S3: after the user takes out the first pen, obtaining the remaining ink percentage of the first pen after the last use, and displaying the remaining ink percentage on the display screen; S4: When the user performs a writing task, the auxiliary camera is used to monitor a movement trajectory of the tip of the first pen on the paper, and the length of the movement trajectory is identified; S5: determining a unit ink consumption percentage of the first pen, calculating a real-time ink consumption percentage of the first pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time, wherein the unit ink consumption percentage is the ink consumption percentage after the tip of the first pen moves on the paper by a unit track length; S6: when the remaining ink percentage is lower than a preset value, inquiring the user about the remaining writing amount percentage, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount percentage, and if so, suggesting the user to continue using the first pen; S7: If not, a second pen for replacing the first pen is determined from other pens in the pen holder according to the characteristics of the first pen, a top view is displayed on the display screen, and the second pen is marked in the top view to guide the user to take out the second pen from the pen holder after the ink of the first pen is used up to continue the writing task; S8: Repeat steps S4 to S7 until the user completes the writing task; Determining a second pen for replacing the first pen from other pens in the pen holder according to the characteristics of the first pen includes: S71: Determine writing characteristics of a first pen, wherein the writing characteristics include ink color and pen tip size; S72: Determine, from other pens in the pen holder, a candidate pen whose writing characteristics have a deviation from the writing characteristics of the first pen less than a preset deviation; the preset deviation includes a preset chromaticity deviation corresponding to the ink color and a preset diameter deviation corresponding to the pen tip diameter; S73: determining the pen holding gesture characteristics of the user, and then determining the corresponding pen holding comfort according to the shape and pen holding gesture characteristics of each candidate pen; S74: Determine the alternative pen with the highest corresponding pen holding comfort as the second pen.

2. The method according to claim 1, characterized in that Determining the unit ink consumption percentage of the first pen, calculating the real-time ink consumption percentage of the habitual pen according to the unit ink consumption percentage of the first pen and the track length, and then updating the remaining ink percentage in real time includes: Obtaining the pen records of all historical pens of the same model as the first pen, wherein the pen records include the usage time interval of the corresponding historical pen and the total length of the movement track corresponding to the historical pen in the usage time interval; For each historical pen, divide 100% by the total length to get the ink consumption efficiency of the historical pen; Calculate the average of the ink consumption efficiency of each historical pen, and obtain the unit ink consumption percentage of the first pen; Each time the length of the newly added track of the first pen is detected, the real-time ink consumption percentage is obtained by multiplying the track length by the unit ink consumption percentage; Subtract the real-time ink consumption percentage from the latest remaining ink percentage to obtain the updated remaining ink percentage.

3. The method according to claim 2, characterized in that The preset value is a preset percentage; inquiring the user about the remaining writing amount ratio, and determining whether the user can complete the writing task with the remaining ink of the first pen according to the remaining writing amount ratio includes: Inquiring the user about the remaining writing volume ratio, so that the user can input the remaining writing volume ratio on the screen; Subtract the remaining writing percentage from 100% to get the completed writing percentage. Retrieve the track length corresponding to the completed writing amount, and then divide the retrieved track length by the proportion of the completed writing amount to obtain the total track length; Calculating a first remaining track length corresponding to the writing task according to the total track length and the remaining writing amount ratio; Calculating the second remaining track length corresponding to the first pen stroke according to the preset value and the unit ink consumption percentage; The first remaining track length and the second remaining track length are compared. If the first track length is greater than the second track length, the user cannot complete the writing task with the remaining ink of the first pen. Otherwise, the user can complete the writing task with the remaining ink of the first pen.

4. The method according to claim 3, characterized in that The first remaining trajectory length is calculated by the following formula: ; in, is the first remaining trajectory length, is the total trajectory length of the writing task, is the proportion of remaining writing volume; The second remaining trajectory length is calculated by the following formula: ; in, is the second remaining trajectory length, is the default value, It is the percentage of unit ink consumption.

5. The method according to claim 3, characterized in that: The alternative pens whose writing characteristics are determined to have a deviation less than a preset deviation from the writing characteristics of the first pen from the other pens in the pen holder include: S721: Select a pen from other pens in the pen holder as a comparison pen; S722: Determine whether the chromaticity deviation between the ink color of the comparison pen and the first pen is less than a preset chromaticity deviation, if not, the comparison pen is not a reserve pen; S723: If yes, determine whether the deviation between the tip diameters of the comparison pen and the first pen is less than a preset diameter deviation; if no, the comparison pen is not a reserve pen; if yes, the comparison pen is a reserve pen; S724: Repeat steps S721 to S723 until all alternative pens are determined.

6. The method according to claim 3, characterized in that Determining the user's pen holding gesture characteristics, and then determining the corresponding pen holding comfort according to the shape and pen holding gesture characteristics of each candidate pen includes: For each candidate pen, generate a three-dimensional pen model of the candidate pen; Capture the user's pen-holding gesture image through the auxiliary camera; Generate a three-dimensional model of the user's hand when holding the pen based on the gesture image; Retrieving the user's historical gesture images, determining the pen holding distance between the pen holding point and the pen tip in each historical gesture image, and calculating the average of the pen holding distances, wherein the pen holding point is the midpoint of the pen cross section where the thumb and index finger are located; Identify the support area on each historical gesture image, and generate a marker area at the corresponding position of the three-dimensional hand model, wherein the support area is the area of ​​the base of the user's hand used to support the pen holder; Identify the target area on the hand 3D model where the support areas with a set number of layers overlap, retain the target area, and remove other support areas; Placing the 3D pen model on the 3D hand model, adjusting the position of the 3D pen model so that the holding distance of the 3D hand model to the holding point of the 3D pen model is equal to the average holding distance, and the pen holder of the 3D pen model is against the 3D hand model; The contact area between the three-dimensional hand model and the three-dimensional pen model is identified, and the overlap rate between the contact area and the target area is determined, and then the holding comfort of the alternative pen is determined according to the overlap rate.

7. The method according to claim 6, characterized in that The overlap ratio between the contact area and the target area is calculated by the following formula: ; in, is the overlap rate, is the total area of ​​the contact region, is the area of ​​the partial contact area that overlaps with the target area; The holding comfort of the alternative pens is calculated by the following formula: The holding comfort of the alternative pens is calculated by the following formula: ； in, For writing comfort, is the overlap rate adjustment coefficient.

8. A smart pen holder, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method of the smart pen holder according to any one of claims 1 to 7.

9. A terminal-readable storage medium, characterized in that: A computer program is stored on the terminal-readable storage medium. When the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method of the smart pen holder according to any one of claims 1 to 7.

Images