A user intention recognition method and system of a scanning pen, an electronic device, and a medium

Abstract

The application provides a user intention recognition method and system of a scanning pen, an electronic device and a medium. When the scanning pen is pressed, the camera takes a picture every preset time, and adds a timestamp on the picture. When the pressing is over, the image information in the pictures is compared based on the timestamp, and the action type of the scanning pen is judged according to the repetition rate of the image information in the pictures. Compared with the existing recognition method, the recognition method of the scanning pen has high recognition accuracy, can accurately recognize the action type of the scanning pen, and accurately judges the user intention.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of scanning equipment, in particular to the field of scanning interaction technology, and more particularly to a user intention recognition method and system for a scanning pen, an electronic device and a readable storage medium. BACKGROUND

[0002] The description of the background of the present application is related to the relevant technology related to the present application, and is only used to illustrate and facilitate the understanding of the application content of the present application, and should not be understood as the applicant's explicit recognition or presumption that it is the prior art on the date of the first filing of the application.

[0003] When the working end of the scanning pen scans the text on books, newspapers or even packaging and other media, it can recognize, store and edit the text through the built-in optical character recognition (OCR) module, which is very convenient in court records, student learning, travel translation and other scenarios.

[0004] For example, Chinese patent CN205003693U discloses an optical scanning pen, which comprises a processor, a camera module for scanning a to-be-scanned object to obtain a character image and transmitting the obtained character image to the processor, an illumination module for providing illumination for the camera module, and a trigger module comprising a trigger mechanism for sending a trigger signal to trigger the processor to enter a working mode when the to-be-scanned object is contacted or at a predetermined distance from the to-be-scanned object; wherein the processor is configured to perform image processing and optical character recognition on the character image to obtain a recognized character code; the optical scanning pen further comprises a wireless module for wirelessly transmitting the recognized character code to a PC or a smart terminal.

[0005] That is, the patent uses distance sensor measurement data to control the opening of the scanning pen, and usually assists in light compensation to interact with the user, and starts scanning by default to determine the user's scanning intention.

[0006] And the scanning content is recognized as characters. However, the similar technology of the scanning pen at present cannot perceive the user's multiple operation intentions. SUMMARY

[0007] The present application provides a user intention recognition method and system for a scanning pen, an electronic device and a readable storage medium.

[0008] The first aspect of the present application provides a user intention recognition method for a scanning pen, characterized in that it comprises the following steps:

[0009] Step S10, listening to whether the scanning pen is pressed;

[0010] Step S20, if the scanning pen is pressed, initialize the camera, ready to take pictures;

[0011] Step S30, the scanning pen continues to listen to whether the user releases the micro switch, if not, then execute steps S40-S50, if yes, then execute step S60;

[0012] Step S40, the scanning pen is pressed during the continuous shooting, and continuously read pictures and exposure time stamp from the camera cache;

[0013] Step S50, the image taken from the cache is numbered, the time stamp is added to each picture, and sent to the server;

[0014] Step S60, send the end flag;

[0015] Step S70, the cloud server processes the obtained shooting pictures.

[0016] Optionally, it also includes steps:

[0017] S80: the cloud processor determines the action type of the scanning pen according to the pictures and time stamps;

[0018] S90: the cloud processor performs corresponding operations according to the action type.

[0019] Optionally, step S80 is specifically,

[0020] S801: when the pen tip displacement of the image information in all pictures with time stamps is below the first threshold value, determine the action type of the scanning pen as "point check";

[0021] Step S90 is specifically,

[0022] S901: perform the first operation according to the "point check" action type.

[0023] Step S80 is specifically,

[0024] Step S802, when the pen tip displacement of the image information in each adjacent time stamp picture in the plurality of pictures arranged in time sequence is within the second threshold value, determine the scanning pen as a "scanning" action;

[0025] Step S90 is specifically,

[0026] Step S902, perform the second operation according to the "scanning" action type.

[0027] Optionally, step S80 is specifically,

[0028] In step S803, when the repetition rate of image information in each picture with adjacent time stamps is within a third threshold, and the repetition rate of image information in pictures with time stamps separated by a preset number of time stamps is within a fourth threshold, the action of the scanning pen is determined as a "circling" action.

[0029] In step S90, the following is specifically implemented:

[0030] In step S903, a third operation is performed according to the "circling" action type.

[0031] Another aspect of the present application provides a recognition system of a scanning pen, comprising:

[0032] A listening module is configured to listen to whether the scanning pen is pressed and send a listening signal;

[0033] A photographing module is configured to control a camera of the scanning pen to start photographing or end photographing according to the listening signal;

[0034] A time module is configured to add a time stamp on each picture;

[0035] A judgment module is configured to determine the action type of the scanning pen according to the picture and the time stamp and send an execution signal; and

[0036] An execution module is configured to perform a corresponding operation according to the execution signal.

[0037] Optionally, the judgment module comprises: a comparison unit configured to compare the repetition rate of image information in the picture according to the time stamp; and

[0038] A generation unit is configured to determine the action type of the scanning pen according to the repetition rate and send the execution signal to the execution module.

[0039] A third aspect of the present application provides a scanning pen, characterized in that the scanning pen comprises the recognition system as described above.

[0040] A fourth aspect of the present application provides an electronic device, characterized in that the electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the recognition method of the scanning pen as described above when executing the computer program.

[0041] A fifth aspect of the present application provides a non-transitory computer readable storage medium, which stores a computer program executable by a processor, and the computer program implements the recognition method of the scanning pen as described in any one of claims 1-5 when executed by the processor.

[0042] The above technical solution of the present application has the following advantages:

[0043] ①Pure online mode; offline NPU or CPU scanning pen production cost, the cost of the user to buy far more than the online solution scanning pen, because the offline scanning pen based on the NPU or CPU of big power, the cost of manufacturing is much higher than the online cost, the more people use the online solution, the lower the cost of the solution.

[0044] ②Depend on the hardware RTOS real-time operating system, especially VxWorks to run the user intention recognition method of scanning pen, VxWorks has hard real-time, determinacy and stability, also has the scalability and safety required by this technology.

[0045] ③With the help of the cloud super strong computing power, thereby reducing the hardware requirements. The traditional scanning pen is based on offline CPU or NPU computing power, and this application is based on cloud GPU server. Because the cloud server has super strong computing power and speed, it has natural advantages in text splicing and character recognition compared with offline recognition scheme. This application takes advantage of the super strong computing power of the cloud to reduce the hardware requirements. Through the powerful cloud GPU cluster computing power, the accuracy of character recognition is greatly improved, and at the same time, since the computing power is in the cloud, the requirements for the front-end hardware are significantly reduced, thereby greatly reducing the cost of the overall scheme.

[0046] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application.

[0047] Additional aspects and advantages of the present application will become apparent from the following description, or will be learned by practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0049] Figure 1 The application scenario diagram of the application system of the scanning pen described in the present application;

[0050] Figure 2 The structure diagram of part of the hardware structure of the scanning pen described in the present application;

[0051] Figure 3 The structure perspective view of part of the hardware structure of the scanning pen described in the present application;

[0052] Figure 4 The main flowchart of the method in which the user side and the server side jointly complete the method in the embodiment of the recognition method of the scanning pen described in the present application;

[0053] Figure 5Another implementation flowchart of the method for identifying the scanning pen according to the application is shown in FIG. 1 1.

[0054] Figure 6 The structure diagram of the identification system of the scanning pen according to the application is shown in FIG. 12.

[0055] Figure 7 The structure diagram of the judging module according to the application is shown in FIG. 13.

[0056] Figure 8 The structure diagram of the electronic device according to the application is shown in FIG. 14.

[0057] Wherein, Figures 1 to 7 The correspondence between the reference signs and the component names in the drawings is as follows:

[0058] The identification system 1000, the listening module 100, the photographing module 200, the time module 300, the judging module 400, the comparison unit 401, the generating unit 402, the execution module 500, the processor 10, the communication interface 20, the storage 30, and the communication bus 40. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purposes, features and advantages of the application, the application will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be practiced without the other ways different from those described herein, and therefore, the scope of protection of the application is not limited by the specific embodiments disclosed below.

[0061] The following discussion provides a plurality of embodiments of the application. Although each embodiment represents a single combination of the application, different embodiments of the application can be replaced or combined, and therefore the application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes A, B and C, and another embodiment includes the combination of B and D, the application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C and D, although the embodiment may not be explicitly described in the following content.

[0062] Figure 1 The application scenario diagram of the application system of the scanning pen according to the application is shown in FIG. 15.

[0063] In Figure 1The user intention recognition method of the applied scanning pen is that the scanning pen is connected with a cloud server, the cloud server runs an operating system, and the scanning pen runs an interactive operation interface corresponding to the operating system of the cloud server.

[0064] In the embodiment of the application, the cloud server is based on a large-scale distributed computing system, adopts a virtualization technology to integrate a large amount of computing power and software resource services. The cloud server runs to judge according to the picture sequence transmitted by the scanning pen, and further recognize the user intention and execute corresponding instructions.

[0065] The scanning pen can run a micro operating system, which provides the most basic mechanisms and functions required by an operating system. Computer chips (embedded systems) usually do not have an operating system. They use code without an operating system to be "hard-coded" to help manage the way code is executed. The application adopts an embedded operating system (RTOS) that meets real-time control requirements, for example, VxWorks is used to run the user intention recognition method of the scanning pen. VxWorks has hard real-time, determinism and stability, and also has scalability and security required by the technology. VxWorks supports predictable task synchronization mechanisms, supports communication between multiple tasks, memory optimization management, and the behavior of the operating system (interrupt delay, task switching, driver delay, etc.) is known and predictable. Real-time clock service + interrupt management service. VxWorks has a high-performance operating system kernel Wind (good real-time performance, scalable) friendly development and debugging environment, good compatibility, and supports multiple development and running environments. Based on this, the interactive operation interface can be run as a software program on the micro operating system, thereby minimizing the demand for software and hardware resources of the scanning pen.

[0066] Figure 2 The structural diagram of part of the hardware structure of the scanning pen described in the application; Figure 3 The structural perspective view of part of the hardware structure of the scanning pen described in the application.

[0067] The main flow of the user intention recognition method of the scanning pen triggering interaction of the application will be described below in combination with the hardware structure of Figure 1 and Figure 2 . Figure 4

[0068] Step S10, listen to whether the scanning pen is pressed.

[0069] This step can specifically include:

[0070] Step S101, the user starts the micro switch;

[0071] For example, in combination with Figure 1 and Figure 2 ​The user presses the micro switch button on the outer shell of the scanning pen, and the micro switch in the scanning pen is pressed due to the mechanical linkage of the micro switch button on the outer shell to the micro switch connecting rod in the scanning pen.

[0072] In step S102, the listening module 100 of the scanning pen captures the level change of the micro switch.

[0073] The threshold level is usually set to determine the user's pressing intention and exclude false operations such as false touch.

[0074] In step S20, if the scanning pen is pressed, the camera is initialized, and the photographing module 200 is ready to take pictures.

[0075] The initialization step includes initializing the websocket link between the scanning pen and the server and preparing to send pictures to the server.

[0076] In step S30, the scanning pen continuously listens to whether the user releases the micro switch, and if not, steps S40-S50 are executed, and if yes, step S60 is executed.

[0077] In step S40, during the pressing of the scanning pen, the photographing module 200 continuously takes pictures and continuously reads pictures and exposure time stamps from the camera cache.

[0078] In step S50, the images taken from the cache are numbered, the time module 300 adds time stamps to each picture, and the pictures are sent to the server.

[0079] In step S60, an end flag is sent.

[0080] In step S70, the cloud server processes the obtained photographed pictures.

[0081] The application can further include the following steps:

[0082] S80: The cloud processor determines the action type of the scanning pen according to the pictures and time stamps.

[0083] S90: The cloud processor performs corresponding operations according to the action type.

[0084] When the scanning pen is pressed, the camera takes a picture every preset time and adds a time stamp to the picture. When the pressing is over, the image information in multiple pictures is compared based on the time stamp, and the action type of the scanning pen is determined according to the repetition rate of the image information in the pictures. Compared with the existing recognition method, the recognition method of the scanning pen has high recognition accuracy, can accurately recognize the action type of the scanning pen, and accurately determines the user's intention.

[0085] It is worth mentioning that the traditional scanning pen is based on offline CPU or NPU computing power, and the application is based on cloud GPU server. Since the cloud server has super strong computing power and speed, it has a natural advantage in text splicing and character recognition compared with the offline recognition scheme. The application takes advantage of the super strong computing power of the cloud to reduce the hardware requirements. Through the powerful cloud GPU cluster computing power, the accuracy of character recognition is greatly improved, and at the same time, since the computing power is in the cloud, the requirements for the front-end hardware are significantly reduced, thereby greatly reducing the cost of the overall scheme.

[0086] The manufacturing cost and user purchase cost of the offline NPU or CPU scanning pen are much higher than those of the online scanning pen. Since the offline scanning pen is based on NPU or CPU with large computing power, the manufacturing cost is much higher than that of the online solution. The more people use the online solution, the lower the cost of the solution.

[0087] The online solution can obtain the latest and most accurate data in real time, and the offline scanning pen needs to update the database to use the latest data.

[0088] As shown in Figure 4 The user intention recognition method of the scanning pen provided by the embodiment of the first aspect of the application comprises the following steps:

[0089] Step S10, listen to whether the scanning pen is pressed.

[0090] This step S10 can specifically include:

[0091] Step S101, the user starts the micro switch;

[0092] As shown in Figure 2 and Figure 3 , the user presses the micro switch button on the scanning pen shell; since the external micro switch button is mechanically connected with the micro switch connecting rod in the scanning pen, the micro switch in the scanning pen is pressed. Of course, it can also be directly set as an electric switch.

[0093] Step S102, the listening module 100 of the scanning pen captures the level change of the micro switch.

[0094] A threshold level is usually set to determine the user's pressing intention and exclude false operations such as accidental touch.

[0095] Step S20, if the scanning pen is pressed, initialize the camera, and the photographing module 200 is ready to take a picture.

[0096] The initialization step includes initializing the websocket link between the scanning pen and the server, and preparing to send pictures to the server.

[0097] Step S30, the judgment module 400 of the scanning pen continuously monitors whether the user releases the micro switch, if yes, steps S40-S50 are executed, if no, step S60 is executed;

[0098] The judgment module 400 of the scanning pen monitors whether the scanning pen is pressed, if the scanning pen is pressed, the camera stops shooting.

[0099] Step S40, during the pressing of the scanning pen, the shooting module 200 continuously shoots and continuously reads the pictures and exposure time stamps from the cache of the camera;

[0100] Step S50, the images taken from the cache are numbered, the time module 300 adds time stamps on each picture and sends to the server;

[0101] The time stamps are added on each picture. The function of the time stamps is to determine the time distribution of the front-end picture taking. In this way, during the later cloud processing, the action type of the scanning pen can be determined according to the pictures and the time stamps, so as to determine the user's intention.

[0102] Step S60, the end flag is sent;

[0103] Step S70, the cloud server processes the obtained shooting pictures.

[0104] Optionally, next, the server side continues to complete the present application, which can further include(not shown in the figure): Figure 4

[0105] S80: the cloud processor determines the action type of the scanning pen according to the pictures and the time stamps;

[0106] S90: the cloud processor performs corresponding operations according to the action type.

[0107] The recognition method provided in the present application, when the scanning pen is pressed, the camera shoots a picture every preset time, and adds a time stamp on the picture, when the pressing is over, the image information in the multiple pictures is compared according to the time stamps, and the action type of the scanning pen is determined according to the repetition rate of the image information in the pictures. Compared with the existing recognition method, the above-mentioned recognition method of the scanning pen has high recognition accuracy, can accurately recognize the action type of the scanning pen, and accurately determine the user's intention.

[0108] As Figure 5 shown, step S80 can be specifically:

[0109] S801: when the pen tip displacement of the image information in all pictures with time stamps is lower than a first threshold value, the action type of the scanning pen is determined as "pointing and checking".

[0110] Specifically, the first threshold value is 80% to 90%. The image information includes text, patterns, colors, and the like.

[0111] Step S90 is specifically:

[0112] S901: Perform a first operation according to the action type of the "pointing" operation.

[0113] When the user performs a "clicking" operation on the same area, all the pictures captured by the camera are basically the same, so when the repetition rate of the image information in all the pictures with time stamps is 80% to 100%, it can be determined that the user performs a "clicking" operation, and a first operation is performed according to the action type of the "clicking" operation. The first operation can be: according to the time stamps, all the pictures are spliced into a clicking picture, the image information in the clicking picture is read, and then the image information is read out by the scanning pen.

[0114] As shown in FIG. 1, in another embodiment of the present application, Figure 5

[0115] Step S80 is specifically:

[0116] Step S802: When the pen tip displacement of the image information in each adjacent time stamp picture in the plurality of pictures arranged in sequence according to the time stamps is within a second threshold value, it is determined that the scanning pen performs a "scanning" operation.

[0117] The second threshold value is 10% to 30%. The image information includes text, patterns, colors, and the like.

[0118] Step S90 is specifically:

[0119] Step S902: Perform a second operation according to the action type of the "scanning" operation.

[0120] When the user performs a sliding "scanning" operation on an area,

[0121] The image information in the pictures with adjacent time stamps captured by the camera is partially overlapped, so when the repetition rate of the image information in the pictures with adjacent time stamps is within 10% to 30%, it can be determined that the user performs a "scanning" operation, and a second operation is performed according to the action type of the "scanning" operation. The second operation can be: according to the time stamps, all the pictures are spliced into a scanning picture, and the scanning picture is displayed on the display connected to the scanning pen.

[0122] As shown in FIG. 1, in one embodiment of the present application, Figure 5

[0123] Step S80 is specifically:

[0124] ​​Step S803, when the repetition rate of the image information in each picture with adjacent time stamps in the plurality of pictures arranged in sequence according to the time stamps is within a third threshold, and the repetition rate of the image information in pictures with two interval preset number of time stamps is within a fourth threshold, it is judged that the action of the scanning pen is a "circle" action. The third threshold is 10% to 30%. The fourth threshold is 80% to 100%. The picture information includes text, patterns, colors and the like.

[0125] Step S90 is specifically:

[0126] Step S903, performing a third operation according to the "circle" action type.

[0127] When the user performs a "circle" operation on a region, first, the image information in pictures with adjacent time stamps captured by the camera partially overlaps, and second, the repetition rate of the image information in pictures with two interval preset number of time stamps is relatively high; therefore, when the repetition rate of the image information in pictures with adjacent time stamps is within 10% to 30%, and the repetition rate of the image information in pictures with two interval preset number of time stamps is within 80% to 100%, it can be judged that the user performs a "circle" operation, and a third operation is performed according to the "circle" action type. The third operation can be marking the picture information in the drawn circle.

[0128] As shown in Figure 6 The recognition system 1000 of the scanning pen provided by the embodiment of the second aspect of the present application includes a listening module 100, a photographing module 200, a time module 300, a judgment module 400 and an execution module 500.

[0129] The listening module 100 is used for listening whether the scanning pen is pressed, and sending a listening signal.

[0130] The photographing module 200 is used for controlling the camera of the scanning pen to start photographing or end photographing according to the listening signal.

[0131] The time module 300 is used for adding a time stamp on each picture.

[0132] The judgment module 400 is used for judging the action type of the scanning pen according to the picture and the time stamp, and sending an execution signal.

[0133] The execution module 500 is used for performing a corresponding operation according to the execution signal.

[0134] The recognition system 1000 provided in the application, when the scanning pen is pressed, the photographing module 200 controls the camera to take a picture every preset time, the time module 300 adds a time stamp on the picture, when the pressing is over, the judging module 400 compares the image information in the pictures according to the time stamp, and judges the action type of the scanning pen according to the repetition rate of the image information in the pictures. Compared with the existing recognition system, the recognition system 1000 of the scanning pen has high recognition accuracy, can accurately recognize the action type of the scanning pen, and accurately judges the user's intention.

[0135] As shown in the embodiment of the application, the judging module 400 comprises a comparison unit 401 and a generation unit 402. Figure 7

[0136] The comparison unit 401 is used to compare the repetition rate of the image information in the pictures according to the time stamp.

[0137] The generation unit 402 is used to judge the action type of the scanning pen according to the repetition rate, and sends an execution signal to the execution module 500.

[0138] When the repetition rate of the image information in all the pictures with time stamps is within 80% to 100%, it can be judged that the user performs a "click" operation, a first operation is performed according to the action type of the "click", the first operation can be: according to the sequence of the time stamps, all the pictures are spliced into a click picture, and the image information in the click picture is read, and then the scanning pen is controlled to read out the image information. When the repetition rate of the image information in the pictures with adjacent time stamps is within 10% to 30%, it can be judged that the user performs a "scanning" operation, a second operation is performed according to the action type of the "scanning", the second operation can be: according to the sequence of the time stamps, all the pictures are spliced into a scanning picture, and the scanning picture is displayed on the display connected with the scanning pen.

[0139] When the repetition rate of the image information in the pictures with adjacent time stamps is within 10% to 30%, and the repetition rate of the image information in the pictures with two interval preset number of time stamps is within 80% to 100%, it can be judged that the user performs a "circle drawing" operation, a third operation is performed according to the action type of the "circle drawing", the third operation can be: marking the image information in the drawn circle.

[0140] The scanning pen provided in the embodiment of the third aspect of the application comprises the recognition system described above.

[0141] As shown in the embodiment of the application, the judging module 400 comprises a comparison unit 401 and a generation unit 402. Figure 8 ​As shown, the electronic device provided by the embodiment of the fourth aspect of the present application can include a processor 10, a communications interface 20, a memory 30 and a communications bus 40, wherein the processor 10, the communications interface 20 and the memory 30 complete mutual communication through the communications bus 40. The processor 10 can invoke the logical instructions in the memory 30 to execute the control method of the scanning pen, which includes: listening whether the scanning pen is pressed. If the scanning pen is pressed, controlling the camera of the scanning pen to take a picture to generate a picture. Adding a time stamp on each picture. The role of the time stamp is to determine the time distribution of the front-end picture taking. Listening whether the scanning pen is ended to be pressed. If the scanning pen is ended to be pressed, controlling the camera to end the picture taking. Judging the action type of the scanning pen according to the picture and the time stamp. Executing the corresponding operation according to the action type.

[0142] In addition, the logical instructions in the memory 30 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0143] On the other hand, the embodiment of the present application also provides a computer program product, which includes a computer program stored in a non-transitory computer readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the control method of the scanning pen provided by the above-mentioned method embodiments, which includes: listening whether the scanning pen is pressed. If the scanning pen is pressed, controlling the camera of the scanning pen to take a picture to generate a picture. Adding a time stamp on each picture. The role of the time stamp is to determine the time distribution of the front-end picture taking. Listening whether the scanning pen is ended to be pressed. If the scanning pen is ended to be pressed, controlling the camera to end the picture taking. Judging the action type of the scanning pen according to the picture and the time stamp. Executing the corresponding operation according to the action type.

[0144] In yet another aspect, the embodiments of the present application also provide a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method of the scanning pen provided by the above-mentioned embodiments, and the method comprises: listening to whether the scanning pen is pressed. If the scanning pen is pressed, controlling the camera of the scanning pen to take a picture to generate a picture. Adding a time stamp on each picture. The role of the time stamp is to determine the time distribution of the front-end picture taking. Listening to whether the scanning pen is ended to be pressed. If the scanning pen is ended to be pressed, controlling the camera to end the picture taking. Judging the action type of the scanning pen according to the picture and the time stamp. Performing corresponding operations according to the action type.

[0145] The apparatus embodiments described above are only schematic, and the units shown as separate units can or can not be physically separate, and the units shown as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. Those skilled in the art can understand and implement without creative labor.

[0146] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0147] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A user intention recognition method of a scanning pen, wherein, the scanning pen is connected with a cloud server, the cloud server runs an operating system, and the scanning pen runs an interactive operation interface corresponding to the operating system of the cloud server; the cloud server is based on a large-scale distributed computing system, adopts a virtualization technology to integrate a large amount of computing power, and provides services for software resources; the cloud server runs to judge according to a picture sequence transmitted by the scanning pen, and then recognize a user intention and execute a corresponding instruction, and the method comprises the following steps: Step S10, listen to whether the scanning pen is pressed; The step specifically comprises: Step S101, the user starts the micro switch; Step S102, the listening module of the scanning pen captures the level change of the micro switch; Step S20, if the scanning pen is pressed, initialize the camera and prepare to take pictures; The initialization step includes initializing the websocket link between the scanning pen and the server, and preparing to send pictures to the server; Step S30, the scanning pen continuously listens to whether the user releases the micro switch, if not, step S40-S50 is executed, if yes, step S60 is executed; Step S40, during the pressing of the scanning pen, continuously shooting and generating pictures, and continuously reading pictures and exposure time stamps from the cache of the camera; Step S50, number the images taken from the cache, add time stamps to each picture, and send them to the server; Add time stamps to each picture to determine the time distribution of the front-end image taking; In the later cloud processing, the action type of the scanning pen is judged according to the pictures and time stamps, so as to judge the user intention; Step S60, send an end flag; Step S70, the cloud server processes the obtained shooting pictures; Step S80: the cloud processor judges the action type of the scanning pen according to the pictures and time stamps; Step S80 specifically comprises, S801: when the repetition rate of image information in all pictures with time stamps is within a first threshold, the action type of the scanning pen is judged as "point check"; Step S80 specifically comprises, Step S803, when the repetition rate of image information in each adjacent time stamp picture in a plurality of pictures arranged in order according to time stamps is within a third threshold, and the repetition rate of image information in pictures with two interval preset number of time stamps is within a fourth threshold, the action of the scanning pen is judged as "circle drawing" action; Step S90: the cloud processor executes a corresponding operation according to the action type; Step S90 specifically comprises, S901: execute a first operation according to the "point check" action type; the first operation is to splice all pictures into a click picture according to the time stamps, read the picture information in the click picture, and then control the scanning pen to read out the picture information; Step S90 specifically comprises, Step S903, execute a third operation according to the "circle drawing" action type, and the third operation is to mark the picture information in the drawn circle.

2. The user intention recognition method of the scanning pen according to claim 1, wherein, Step S80 specifically comprises, In step S802, when the repetition rate of the image information in each picture with adjacent time stamps in the plurality of pictures arranged in sequence according to the time stamps is within the second threshold, it is determined that the action of the scanning pen is a "scanning" action. Step S90 specifically includes, In step S902, a second operation is performed according to the action type of "scanning", and the second operation is to splice all pictures into a scanning picture according to the time stamps, and display the scanning picture on the display connected to the scanning pen.

3. An electronic device, comprising: The computer program stored in the memory and executable on the processor, when the processor executes the computer program, implements the user intention recognition method of the scanning pen according to any one of claims 1-2.

4. A non-transitory computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the user intention recognition method of the scanning pen according to any one of claims 1-2.

Images