User intention recognition method and system of scanning and reading pen, electronic equipment and medium

Abstract

The invention relates to the technical field of intention recognition, and discloses a user intention recognition method and system for a scanning and reading pen, electronic equipment and a medium, and the method comprises the steps: monitoring whether the scanning and reading pen is pressed or not, and when the scanning and reading pen is pressed, initializing a camera, and preparing for photographing; when the scanning and reading pen is pressed, continuously shooting and generating pictures according to a preset frame difference, and continuously reading the pictures and timestamps from the cache of the camera; numbering the cached images taken out from the cache, and adding a timestamp on each cached image to obtain a numbered scanning pen image; performing scanning and reading pen motion analysis according to the numbered scanning and reading pen image to obtain scanning and reading pen motion characteristics; performing user intention analysis according to the scanning pen motion characteristics to obtain a real intention of the user; and uploading the serial number scanning pen image and the real intention of the user to a cloud server. According to the method, the user intention recognition accuracy of the scanning and reading pen can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of user intention recognition, and in particular to a method, system, electronic device and medium for recognizing user intention of a scanning pen. Background Art

[0002] At present, the scanning pen is an intelligent optical assisted learning device based on image recognition detection technology, with a built-in high-precision camera, high-speed image capture technology, ultra-clear resolution, and high-definition display. By shooting words, sentences or other texts through the shooting unit of the scanning pen, the text can be quickly and accurately scanned and read through OCR recognition technology, and the scanned content can be pronounced. As the functions of scanning pens on the market are increasing, the operation process is becoming more and more complicated, and the instructions on the existing scanning pens are generally switched through the buttons on the scanning pen or the menus set on the interface of the mobile phone APP. However, the scanning pen needs to keep the word selection state at all times, and the operation process is cumbersome and inhumane. Users cannot accurately remember the current functions of the scanning pen, which makes it easy for users to operate incorrectly.

[0003] In one prior art, a method for identifying user intentions of a scanning pen is disclosed. When the scanning pen is pressed, the camera takes a picture at a preset time interval and adds a timestamp to the picture. When the pressing ends, the image information in multiple pictures is compared based on the timestamp, and the action type of the scanning pen is determined according to the repetition rate of the image information of the pictures. For example: when the tip displacement of the image information in all pictures with timestamps is lower than the first threshold, the action type of the scanning pen is determined to be "point check"; in multiple pictures arranged in sequence according to timestamps, when the repetition rate of the image information in each picture with adjacent timestamps is within the third threshold, and the repetition rate of the image information in two pictures with a preset number of timestamps is within the fourth threshold, the action of the scanning pen is determined to be a "circle drawing" action.

[0004] However, in the prior art, the user intention is judged only by the action and the preset height threshold, without judging according to the specific motion trajectory, resulting in low accuracy of user intention recognition. Summary of the invention

[0005] The present invention provides a method, system, electronic device and medium for identifying user intention of a scanning pen, so as to improve the accuracy of identifying user intention.

[0006] In a first aspect, in order to solve the above technical problem, the present invention provides a method for identifying user intention of a scanning pen, comprising: Monitor whether the scanning pen is pressed. When the scanning pen is pressed, initialize the camera and prepare to take pictures. While the scanning pen is pressed, it continuously captures and generates images according to the preset frame difference, and continuously reads images and timestamps from the camera's cache; Numbering the cached images taken out from the cache, adding a timestamp to each of the cached images, and obtaining a numbered scanning pen image; Performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features; Performing user intention analysis based on the scanning pen motion characteristics to obtain the user's real intention; The number scanning pen image and the user's real intention are uploaded to the cloud server.

[0007] In an optional implementation, before performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features, the method further includes: The total shooting time difference is calculated by the following formula: ; in, is the total shooting time difference, is the maximum timestamp, is the minimum timestamp; When the total shooting time difference is greater than the preset maximum shooting threshold, the shooting is deemed invalid and no subsequent operation is performed; When the total shooting time difference is less than the preset minimum shooting threshold, it is considered that the shooting time is too short and no subsequent operation is performed; When the total shooting time difference is less than or equal to a preset maximum shooting threshold and greater than or equal to a preset minimum shooting threshold, subsequent operations are performed.

[0008] In an optional implementation, performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features includes: Perform scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates; The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; The scanning pen movement direction is calculated by the following formula: ; in, for i No. Scan the image movement direction of the pen, for i +The vertical coordinate of the center of the No. 1 scanning pen, for i The vertical coordinate of the center of the scanning pen, for i+ The horizontal coordinate of the center of scanning pen No. 1, for i The horizontal coordinate of the center of the scanning pen; The scanning pen movement angle is calculated by the following formula: ; in, for i No. Scanning pen image movement angle; The scanning pen movement angle is calculated by the following formula: ; in, express i No. Scan pen image movement distance.

[0009] In an optional implementation, the step of performing scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates includes: grayscale the numbered scanning pen image to obtain a scanning pen grayscale image; Performing contour detection on the grayscale image of the scanning pen to obtain a contour of the scanning pen; The minimum circumscribed circle is calculated for the scanning pen contour, and the center coordinates of the minimum circumscribed circle are used as the center coordinates of the scanning pen.

[0010] In an optional implementation, performing user intention analysis according to the scanning pen motion characteristics to obtain the user's real intention includes: The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; When the scanning pen movement distance is less than a preset distance threshold, the scanning pen image with a number corresponding to the scanning pen movement distance is determined to be an invalid image; When the movement distance of the scanning pen is greater than or equal to the preset distance threshold, the movement direction and the movement angle of the scanning pen are continuously judged, including: When the scanning pen movement angles corresponding to the scanning pen images with consecutive numbers exceeding the preset rotation number threshold are greater than the preset rotation angle threshold, it is determined that the user's real intention is "rotation"; When the scanning pen moves in a preset left-slide angle range, it is determined that the user's real intention is "left-slide"; When the scanning pen moves in a preset right-slide angle range, it is determined that the user's real intention is "right-slide".

[0011] In an optional implementation, after analyzing the user intention according to the scanning pen motion characteristics to obtain the user's real intention, the method further includes: When the user's real intention is "rotation", the corresponding function of the scanning pen is "mode switching"; When the user's real intention is "swipe left", the corresponding function of the scanning pen is "translate content"; When the user's real intention is "right swipe", the corresponding function of the scanning pen is "reading content aloud".

[0012] In an optional implementation, after performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features, the method further includes: Perform scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates; Perform interpolation processing according to the center coordinates of the scanning pen to obtain a motion trajectory curve; The motion trajectory of the scanning pen is drawn according to the motion trajectory curve.

[0013] In a second aspect, the present invention provides a user intention recognition system for a scanning pen, comprising: The monitoring module is used to monitor whether the scanning pen is pressed. When the scanning pen is pressed, the camera is initialized to prepare for taking pictures. The shooting module is used to continuously shoot and generate pictures according to a preset frame difference while the scanning pen is pressed, and continuously read pictures and timestamps from the camera's cache; A numbering module, used to number the cached images taken out from the cache, add a timestamp to each of the cached images, and obtain a numbered scanning pen image; A motion analysis module, used for performing a scanning pen motion analysis according to the numbered scanning pen image to obtain a scanning pen motion feature; An intention analysis module, used to analyze the user's intention according to the scanning pen movement characteristics to obtain the user's real intention; The data uploading module is used to upload the image of the serial number scanning pen and the real intention of the user to the cloud server.

[0014] In a third aspect, the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for identifying user intention of a scanning pen as described in any one of the above is implemented.

[0015] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for identifying user intentions of a scanning pen.

[0016] Compared with the prior art, the present invention has the following beneficial effects: the present invention discloses a method, system, electronic device and medium for identifying the user intention of a scanning pen, the method comprising: monitoring whether the scanning pen is pressed, and when the scanning pen is pressed, initializing the camera and preparing to take pictures; during the period when the scanning pen is pressed, continuously shooting and generating pictures according to a preset frame difference, and continuously reading pictures and timestamps from the cache of the camera; numbering the cache images taken out of the cache, adding a timestamp to each of the cache images, and obtaining a numbered scanning pen image; performing scanning pen motion analysis according to the numbered scanning pen images to obtain scanning pen motion features; performing user intention analysis according to the scanning pen motion features to obtain the user's true intention; uploading the numbered scanning pen image and the user's true intention to a cloud server. By monitoring the pressing action of the scanning pen and combining the continuously shot images, the scanning behavior of the user can be accurately captured, while ensuring that no shooting behavior is performed when there is no pressing, thereby avoiding the waste of storage resources. By analyzing the motion features of the scanning pen, the user's intentional operation and unintentional touch can be distinguished, and the influence of misoperation on the recognition of user intention can be reduced.

[0017] Specifically, when the total shooting time difference is greater than the preset maximum shooting threshold, the shooting is deemed invalid and no subsequent operations are performed; when the total shooting time difference is less than the preset minimum shooting threshold, the shooting is deemed too short and no subsequent operations are performed. By setting the maximum shooting threshold and the minimum shooting threshold, unnecessary waste of computing and storage resources can be avoided for shots that are too short or too long, thereby improving system efficiency. This includes the possibility that users may make mistakes when operating the scanning pen, such as accidentally pressing or pressing for a long time. By setting the threshold, the impact of these erroneous operations on the system can be reduced.

[0018] Further, the scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, scanning pen movement distance; when the scanning pen movement distance is less than a preset distance threshold, the numbered scanning pen image corresponding to the scanning pen movement distance is determined to be an invalid image; when the scanning pen movement distance is greater than or equal to the preset distance threshold, the scanning pen movement direction and scanning pen movement angle are continuously judged, including: when the scanning pen movement angle corresponding to the numbered scanning pen images with consecutive numbers exceeding a preset rotation number threshold is greater than a preset rotation angle threshold, the user's real intention is judged to be "rotation"; when the scanning pen movement direction is within a preset left sliding angle range, the user's real intention is judged to be "left sliding"; when the scanning pen movement direction is within a preset right sliding angle range, the user's real intention is judged to be "right sliding". By setting a preset distance threshold, invalid frames in the scanning pen image can be eliminated, reducing invalid calculations and improving algorithm efficiency. At the same time, by combining the scanning pen movement angle and scanning movement direction for dynamic analysis, user actions can be accurately identified, thereby improving the accuracy of the scanning pen's user intent recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of a method for identifying user intention of a scanning pen provided by a first embodiment of the present invention; Figure 2 2 is a schematic diagram of the structure of a user intention recognition system of a scanning pen provided in a second embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, the scanning pen is an intelligent optical assisted learning device based on image recognition detection technology. It has a built-in high-precision camera, high-speed image capture technology, ultra-clear resolution, and a high-definition display. By shooting words, sentences or other texts through the shooting unit of the scanning pen, the text can be quickly and accurately scanned and read through OCR recognition technology, and the scanned content can be pronounced. As the functions of scanning pens on the market are increasing, the operation process is becoming more and more complicated, and the instructions on the existing scanning pens are generally switched through the buttons on the scanning pen or the menus set on the interface of the mobile phone APP. However, the scanning pen needs to keep the word selection state at all times, and the operation process is cumbersome and inhumane. Users cannot accurately remember the current functions of the scanning pen, which makes it easy for users to make mistakes.

[0022] In one prior art, a method for identifying user intentions of a scanning pen is disclosed. When the scanning pen is pressed, the camera takes a picture at a preset time interval and adds a timestamp to the picture. When the pressing ends, the image information in multiple pictures is compared based on the timestamp, and the action type of the scanning pen is determined according to the repetition rate of the image information of the pictures. For example: when the tip displacement of the image information in all pictures with timestamps is lower than the first threshold, the action type of the scanning pen is determined to be "point check"; in multiple pictures arranged in sequence according to timestamps, when the repetition rate of the image information in each picture with adjacent timestamps is within the third threshold, and the repetition rate of the image information in two pictures with a preset number of timestamps is within the fourth threshold, the action of the scanning pen is determined to be a "circle drawing" action.

[0023] However, in the prior art, the user intention is judged only by the action and the preset height threshold, without judging according to the specific motion trajectory, resulting in low accuracy of user intention recognition.

[0024] To solve the above problems, refer to Figure 1 The first embodiment of the present invention provides a method for identifying user intention of a scanning pen, comprising the following steps: S11, monitoring whether the scanning pen is pressed, when the scanning pen is pressed, initializing the camera and preparing to take pictures; S12, while the scanning pen is pressed, continuously shooting and generating pictures according to a preset frame difference, and continuously reading pictures and timestamps from the camera's cache; S13, numbering the cached images taken out from the cache, adding a timestamp to each of the cached images, and obtaining a numbered scanning pen image; S14, performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features; S15, analyzing the user's intention according to the scanning pen movement characteristics to obtain the user's real intention; S16, uploading the serial number scanning pen image and the user's real intention to a cloud server.

[0025] In step S11, it is monitored whether the scanning pen is pressed. When the scanning pen is pressed, the camera is initialized to prepare for taking pictures.

[0026] In one embodiment, a pressure sensor or a touch switch is integrated inside the scanning pen to detect the user's pressing action. This method has a background service, which requires writing code to interact with the pressure sensor and the touch switch to continuously monitor the pressure sensor signal from the scanning pen. When the scanning pen is detected to be pressed, the camera API or SDK is called to perform initialization operations. This includes starting the camera application, setting the camera resolution, setting the camera focus mode, etc. Set the preview screen to ensure that the user can see the real-time image of the camera through the display screen of the scanning pen or other connected display devices. After completing the camera initialization, enter the photo preparation state, waiting for further instructions from the user or automatically triggering the photo.

[0027] In step S12, while the scanning pen is pressed, the image is continuously captured and generated according to the preset frame difference, and the image and time stamp are continuously read from the cache of the camera.

[0028] In one embodiment, a frame rate is set, which determines how many pictures are taken per second. Depending on the application scenario, different frame rates can be selected, such as a frame rate of 30 frames per second. The present invention is not limited to this. For example: in a fast reading recognition scenario, the user needs to scan the text continuously and quickly. In this case, a higher frame rate, such as 30 frames per second, can be selected; for children's education applications, the selection of the frame rate needs to take into account that children will not write or draw at a steady speed. Therefore, a medium frame rate, such as 15 frames per second, can be selected. Each time a picture is taken, the timestamp of the picture is recorded at the same time. The timestamp is the current time of the system. A memory cache is used to store the taken pictures and their corresponding timestamps. However, the memory cache is fast but has limited capacity and is suitable for short-term storage; the present invention defines a read function that is responsible for reading the latest picture and timestamp from the cache. For the memory cache, the data structure in the memory can be directly accessed.

[0029] In step S13, the cached images taken out from the cache are numbered, and a timestamp is added to each of the cached images to obtain a numbered scanning pen image.

[0030] In one embodiment, the system assigns a unique number to each frame of image retrieved from the cache. This numbering is performed in the order in which the images are retrieved and can be a continuous sequence of numbers. For example, the first image is numbered 1, the second is 2, and so on. For each numbered image, the system obtains a corresponding timestamp, which indicates the specific time when the image was taken or processed. After obtaining the timestamp, the system attaches it to the corresponding image. This timestamp is used as an attribute or metadata of the image for subsequent analysis and processing. The timestamp can be directly embedded in the EXIF ​​data of the image file. The scanning pen image with a timestamp and number is a numbered scanning pen image.

[0031] In one embodiment, before performing scanning pen motion analysis according to the numbered scanning pen image to obtain the scanning pen motion feature, the method further includes: The total shooting time difference is calculated by the following formula: ; in, is the total shooting time difference, is the maximum timestamp, is the minimum timestamp; When the total shooting time difference is greater than the preset maximum shooting threshold, the shooting is deemed invalid and no subsequent operation is performed; When the total shooting time difference is less than the preset minimum shooting threshold, it is considered that the shooting time is too short and no subsequent operation is performed; When the total shooting time difference is less than or equal to a preset maximum shooting threshold and greater than or equal to a preset minimum shooting threshold, subsequent operations are performed.

[0032] It is worth noting that if the shooting time is too long, this is due to improper user operation (for example, pressing the scanning pen for a long time without actual scanning action) or device failure. In this case, the captured data is inaccurate, so it is considered invalid shooting and no subsequent analysis is performed. If the shooting time is too short, this is because the user pressed the scanning pen for too short a time (for example, the user touched it by mistake). In this case, it is also considered invalid shooting and no subsequent analysis is performed. The preset maximum shooting threshold and minimum shooting threshold are analyzed through normal usage scenarios. For a normal scanning pen used to scan a whole line of text, it takes 5 seconds, so the maximum shooting threshold can add a redundancy to the normal behavior, such as 10 seconds. This method does not limit this. At the same time, the maximum shooting threshold can provide multiple gears, which can be selected and set by the user to meet the needs of different users. The minimum shooting threshold can be set to 1 second. For normal scanning pen behavior, the shortest is 2 seconds, and a redundancy time is also added. In general, the maximum shooting threshold can be set to 1.5 to 2 times of the normal longest operation, and the minimum shooting threshold can be set to 0.5 times of the normal shortest operation. In actual use, further adjustments are made according to the performance of the scanning pen.

[0033] In step S14, a scanning pen motion analysis is performed according to the numbered scanning pen image to obtain a scanning pen motion feature.

[0034] In one implementation, the scanning pen center is identified on the numbered scanning pen image to obtain the scanning pen center coordinates; The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; The scanning pen movement direction is calculated by the following formula: ; in, for i No. Scan the image movement direction of the pen, for i +The vertical coordinate of the center of the No. 1 scanning pen, for i The vertical coordinate of the center of the scanning pen, for i+ The horizontal coordinate of the center of scanning pen No. 1, for i The horizontal coordinate of the center of the scanning pen; The scanning pen movement angle is calculated by the following formula: ; in, for i No. Scanning pen image movement angle; The scanning pen movement angle is calculated by the following formula: ; in, express i No. Scan pen image movement distance.

[0035] In one implementation, the numbered scanning pen image is grayed to obtain a scanning pen gray image; contour detection is performed on the scanning pen gray image to obtain a scanning pen contour; the minimum circumscribed circle is calculated for the scanning pen contour, and the center coordinates of the minimum circumscribed circle are used as the center coordinates of the scanning pen.

[0036] It is worth noting that the purpose of converting color images to grayscale images is to reduce the amount of data and simplify the subsequent processing process. The method directly uses the grayscale conversion formula of the RGB channel, for example: .in is the grayscale value, R, G, B are the values ​​of the red, green, and blue channels respectively. Then the Canny edge detection algorithm is used to identify the edges in the image. These edges constitute the outline of the scanning pen. Then the Hough circle transform can be used to find the smallest circle that can contain all the contour points. Once the smallest circumscribed circle is determined, its center coordinates can be directly obtained from the algorithm results. This coordinate represents the center position of the scanning pen in the image. The scanning pen center coordinates are equivalent to a reference point of the scanning pen, which is used for subsequent motion feature analysis.

[0037] In one embodiment, the Canny edge detection algorithm is an edge detection method, which is implemented by the following steps: the image is converted into a grayscale image, and then a Gaussian filter is applied to smooth the image and reduce noise; the gradient magnitude and direction of the image are calculated using the Sobel operator. The gradient magnitude indicates the strength of the edge, and the gradient direction indicates the direction of the edge. Then, non-maximum suppression is performed, that is, in the gradient direction, only when the gradient magnitude of the point is greater than that of its neighboring points, the point is retained as an edge candidate; a double threshold method is applied to determine the candidate edges. First, a high threshold is used to identify strong edges, and then a low threshold is used to determine weak edges. Points with edge strengths higher than the high threshold are determined as edge points, and points with edge strengths lower than the low threshold are excluded. Points with edge strengths between the two thresholds are retained only if they are connected to strong edge points.

[0038] In one embodiment, the Hough circle transform is a feature extraction technique for detecting circles in an image. The basic principle is as follows: for each point, all candidate center positions are calculated based on its gradient direction and radius, and voted in the parameter space; the center and radius are determined by counting the voting results in the parameter space. The center of the circle is the intersection of the normal lines of the circle, and the radius is the distance from the center of the circle to the circumference. A threshold is set. If the number of intersecting straight lines at a point is greater than this threshold, the intersection point is considered to be the center of the circle.

[0039] In one implementation, the scanning pen center is identified on the numbered scanning pen image to obtain the scanning pen center coordinates; interpolation processing is performed based on the scanning pen center coordinates to obtain a motion trajectory curve; and the scanning pen motion trajectory is drawn based on the motion trajectory curve.

[0040] It is worth noting that the center coordinates of the scanning pen are extracted from the numbered scanning pen image and then polynomial interpolation is performed. The following polynomial is used: ; For the acquired scanning pen center, solve the coefficients separately to get the polynomial equation of each segment. Then connect the overall curve to get the motion trajectory curve. If the trajectory is close to a straight line and the speed does not change much, it means that the user is performing regular scanning or reading. If the trajectory presents a curve and changes slowly, it means that the user is browsing or selecting text. If the trajectory has multiple round trips in a short period of time, it means that the user is reviewing or marking key points. If the trajectory forms a spiral or circle, it means that the user is performing specific operations, such as zooming in, zooming out, or selecting specific functions.

[0041] In step S15, the user intention is analyzed according to the scanning pen motion characteristics to obtain the user's real intention.

[0042] In one embodiment, the scanning pen movement feature includes: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; when the scanning pen movement distance is less than a preset distance threshold, the numbered scanning pen image corresponding to the scanning pen movement distance is determined to be an invalid image; when the scanning pen movement distance is greater than or equal to the preset distance threshold, the scanning pen movement direction and the scanning pen movement angle are continuously judged, including: when the scanning pen movement angle corresponding to the numbered scanning pen images that are consecutively numbered and exceed a preset rotation number threshold is greater than a preset rotation angle threshold, the user's real intention is judged to be "rotation"; when the scanning pen movement direction is within a preset left sliding angle range, the user's real intention is judged to be "left sliding"; when the scanning pen movement direction is within a preset right sliding angle range, the user's real intention is judged to be "right sliding". When the user's real intention is "rotate", the corresponding function of the scanning pen is "switch mode"; when the user's real intention is "swipe left", the corresponding function of the scanning pen is "translate content"; when the user's real intention is "swipe right", the corresponding function of the scanning pen is "read content aloud".

[0043] It is worth noting that the scanning pen movement direction refers to the movement direction of the scanning pen on the plane, which can be expressed as an angle. The scanning pen movement angle refers to the rotation angle of the scanning pen during the movement process. The scanning pen movement distance refers to the actual distance the scanning pen moves within a period of time. By defining an angle range for left swipe (for example, -45° to -135°), if the scanning pen movement direction is within this range, it can be considered that the user's true intention is "left swipe". By defining an angle range for right swipe (for example, 45° to 135°), if the scanning pen movement direction is within this range, it can be considered that the user's true intention is "right swipe". Since there are continuous scanning pen images, such as 30 frames, but not every frame is helpful for the scanning pen movement feature, invalid frames can be eliminated by setting a distance threshold to reduce subsequent operations. Too small a movement distance means that the user is hesitating, which is not helpful for user intention recognition. The rotation angles of multiple consecutive frame rates are all greater than the preset rotation threshold. This is the most direct way to analyze the "rotation" intention, and the specific threshold setting should be determined based on the previous frame rate. Here is a reference. For example, when the frame rate is set to 30 frames, it means that the interval between adjacent images is 0.03 seconds. At this time, it can be set to 10 consecutive frames. The movement angle of the scanning pen is greater than 20 degrees. Here, a redundancy is also given for the scanning pen action. However, since normal operation will not rotate 200 degrees in 0.3 seconds, this redundant range is reasonable.

[0044] It is worth noting that the scanning pen has built-in functions such as reading aloud, translating, storing, and recognizing. By rotating the scanning pen, the user can quickly switch the working mode of the scanning pen. For example, switch from ordinary scanning mode to translation mode or reading aloud mode. This design allows the user to quickly switch functions through a simple rotation action. This is also the purpose of the method to accurately identify the user's intention of the scanning pen. By sliding left, the user can trigger the translation function of the scanning pen. For example, the user can slide the scanning pen to the left on the text, and the scanning pen will automatically recognize and translate the selected text. By sliding right, the user can trigger the reading function of the scanning pen. For example, the user can slide the scanning pen to the right on the text, and the scanning pen will automatically recognize and read aloud the selected text. The correspondence between the user's true intention and the working mode of the scanning pen can be adjusted according to user needs, and the present invention is not limited.

[0045] In step S16, the serial number scanning pen image and the user's real intention are uploaded to a cloud server.

[0046] In one embodiment, the number scanning pen image is stored in JPEG format, and the user's intention is converted into a string format. The uploaded data is stored in a database of a cloud server.

[0047] It is worth noting that the present invention is based on a children's education intelligent platform and belongs to the data input link. The method is used to identify the user intention of the scanning pen to obtain accurate input data. The children's education intelligent platform is a comprehensive system that integrates multiple technologies and aims to improve the efficiency and quality of children's education. The data input link of the platform accurately captures the user's scanning behavior through the user intention recognition technology of the scanning pen, reduces misoperation, and obtains accurate input data. The data processing link includes an intelligent recommendation system for children's education content based on knowledge graphs, which analyzes children's learning data and preferences and recommends personalized education content to children; and an adaptive adjustment system for the platform interaction interface based on image recognition, which identifies the environment and equipment when children interact with the platform, automatically adjusts the interaction interface, and improves the user experience. In addition, there is an intelligent education platform interaction system based on children's voice behavior analysis, which understands children's language habits and expressions and provides a natural and friendly interactive experience. The data output link analyzes children's learning performance and progress through a children's mathematical thinking training effect evaluation system based on data mining, evaluates the effect of the education platform, and helps educators and parents adjust education strategies in a timely manner. Overall, the platform covers the entire process from data input to processing to output, providing children with a personalized, interactive and measurable educational environment.

[0048] In summary, the present invention discloses a method for identifying user intentions of a scanning pen, aiming to improve the accuracy of user intention identification of the scanning pen. By monitoring the pressing action of the scanning pen and combining the continuously shot images, the scanning behavior of the user is accurately captured. Specifically, when the scanning pen is pressed, the system initializes the camera to prepare for taking pictures; while the scanning pen is pressed, it continuously shoots and generates pictures according to the preset frame difference, and continuously reads pictures and timestamps from the cache of the camera. The system numbers the cache images taken out from the cache, and adds a timestamp to each cache image to obtain a numbered scanning pen image. Subsequently, the system performs scanning pen motion analysis based on the numbered scanning pen image to obtain scanning pen motion features, including motion direction, motion angle, and motion distance. By analyzing these features, the system can distinguish between intentional operations and unintentional touches of the user, and reduce the impact of misoperation on user intention identification. In terms of technical implementation, the present invention analyzes the motion features of the scanning pen through image processing technology, including grayscale processing, contour detection, minimum circumscribed circle calculation, and other steps. First, the image of the numbered scanning pen is grayed to obtain the gray image of the scanning pen; then, the contour of the gray image is detected to obtain the contour of the scanning pen; finally, the minimum circumscribed circle is calculated for the contour of the scanning pen, and the center coordinates of the minimum circumscribed circle are used as the center coordinates of the scanning pen. These steps accurately capture and analyze the motion characteristics of the scanning pen, thereby improving the accuracy of user intention recognition. The present invention provides an innovative method for user intention recognition of a scanning pen by monitoring the pressing action of the scanning pen, continuously shooting images, analyzing motion characteristics and user intentions, and executing corresponding functions. This method can improve the accuracy of user intention recognition.

[0049] Reference Figure 2 A second embodiment of the present invention provides a user intention recognition system for a scanning pen, comprising: The monitoring module is used to monitor whether the scanning pen is pressed. When the scanning pen is pressed, the camera is initialized to prepare for taking pictures. The shooting module is used to continuously shoot and generate pictures according to a preset frame difference while the scanning pen is pressed, and continuously read pictures and timestamps from the camera's cache; A numbering module, used to number the cached images taken out from the cache, add a timestamp to each of the cached images, and obtain a numbered scanning pen image; A motion analysis module, used for performing a scanning pen motion analysis according to the numbered scanning pen image to obtain a scanning pen motion feature; An intention analysis module, used to analyze the user's intention according to the scanning pen movement characteristics to obtain the user's real intention; The data uploading module is used to upload the image of the serial number scanning pen and the real intention of the user to the cloud server.

[0050] Preferably, the monitoring module is used to: Monitor whether the scanning pen is pressed. When the scanning pen is pressed, initialize the camera and prepare to take pictures.

[0051] Preferably, the shooting module is used for: While the scanning pen is pressed, it continuously captures and generates images according to the preset frame difference, and continuously reads images and timestamps from the camera's cache.

[0052] Preferably, the numbering module is used to: The cached images taken out from the cache are numbered, and a timestamp is added to each of the cached images to obtain a numbered scanning pen image.

[0053] Preferably, the motion analysis module is used to: The scanning pen motion analysis is performed according to the numbered scanning pen image to obtain the scanning pen motion features, including: Performing scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates includes: grayscale the numbered scanning pen image to obtain a scanning pen grayscale image; Performing contour detection on the grayscale image of the scanning pen to obtain a contour of the scanning pen; The minimum circumscribed circle is calculated for the scanning pen contour, and the center coordinates of the minimum circumscribed circle are used as the center coordinates of the scanning pen.

[0054] The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; The scanning pen movement direction is calculated by the following formula: ; in, for i No. Scan the image movement direction of the pen, for i +The vertical coordinate of the center of the No. 1 scanning pen, for i The vertical coordinate of the center of the scanning pen, for i+ The horizontal coordinate of the center of scanning pen No. 1, for i The horizontal coordinate of the center of the scanning pen; The scanning pen movement angle is calculated by the following formula: ; in, for i No. Scanning pen image movement angle; The scanning pen movement angle is calculated by the following formula: ; in, express i No. Scan pen image movement distance.

[0055] Preferably, before performing scanning pen motion analysis according to the numbered scanning pen image to obtain the scanning pen motion feature, the method further includes: The total shooting time difference is calculated by the following formula: ; in, is the total shooting time difference, is the maximum timestamp, is the minimum timestamp; When the total shooting time difference is greater than the preset maximum shooting threshold, the shooting is deemed invalid and no subsequent operation is performed; When the total shooting time difference is less than the preset minimum shooting threshold, it is considered that the shooting time is too short and no subsequent operation is performed; When the total shooting time difference is less than or equal to a preset maximum shooting threshold and greater than or equal to a preset minimum shooting threshold, subsequent operations are performed.

[0056] Preferably, after performing scanning pen motion analysis on the numbered scanning pen image to obtain scanning pen motion features, the method further includes: performing scanning pen center identification on the numbered scanning pen image to obtain scanning pen center coordinates; performing interpolation processing on the scanning pen center coordinates to obtain a motion trajectory curve; and drawing the scanning pen motion trajectory according to the motion trajectory curve.

[0057] Preferably, the intention analysis module is used to: Performing user intention analysis based on the scanning pen motion characteristics to obtain the user's real intention includes: The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, scanning pen movement distance; when the scanning pen movement distance is less than a preset distance threshold, the numbered scanning pen image corresponding to the scanning pen movement distance is determined to be an invalid image; when the scanning pen movement distance is greater than or equal to the preset distance threshold, the scanning pen movement direction and the scanning pen movement angle are continuously judged, including: when the scanning pen movement angles corresponding to the numbered scanning pen images that are consecutively numbered and exceed a preset rotation number threshold are greater than a preset rotation angle threshold, the user's real intention is judged to be "rotation"; when the scanning pen movement direction is within a preset left sliding angle range, the user's real intention is judged to be "left sliding"; when the scanning pen movement direction is within a preset right sliding angle range, the user's real intention is judged to be "right sliding".

[0058] Preferably, after performing user intention analysis according to the scanning pen movement characteristics to obtain the user's true intention, the method further includes: when the user's true intention is "rotation", the corresponding function of the scanning pen is "mode switching"; when the user's true intention is "left swipe", the corresponding function of the scanning pen is "content translation"; when the user's true intention is "right swipe", the corresponding function of the scanning pen is "content reading".

[0059] Preferably, the data uploading module is used to: The number scanning pen image and the user's real intention are uploaded to the cloud server.

[0060] It should be noted that the user intention recognition system of a scanning pen provided in an embodiment of the present invention is used to execute all the process steps of the user intention recognition method of a scanning pen in the above embodiment, and the working principles and beneficial effects of the two correspond one to one, so they will not be repeated here.

[0061] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a data upload program. When the processor executes the computer program, the steps in the above-mentioned user intention recognition method embodiments of the scanning pen are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the data uploading module.

[0062] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0063] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0064] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0065] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0066] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0067] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0068] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying user intention of a scanning pen, characterized in that: include: Monitor whether the scanning pen is pressed. When the scanning pen is pressed, initialize the camera and prepare to take pictures. While the scanning pen is pressed, it continuously captures and generates images according to the preset frame difference, and continuously reads images and timestamps from the camera's cache; Numbering the cached images taken out from the cache, adding a timestamp to each of the cached images, and obtaining a numbered scanning pen image; Performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features; Performing user intention analysis based on the scanning pen motion characteristics to obtain the user's real intention; The number scanning pen image and the user's real intention are uploaded to the cloud server.

2. The method for identifying user intention of a scanning pen according to claim 1, characterized in that: Before performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features, the method further includes: The total shooting time difference is calculated by the following formula: ； in, is the total shooting time difference, is the maximum timestamp, is the minimum timestamp; When the total shooting time difference is greater than the preset maximum shooting threshold, the shooting is deemed invalid and no subsequent operation is performed; When the total shooting time difference is less than the preset minimum shooting threshold, it is considered that the shooting time is too short and no subsequent operation is performed; When the total shooting time difference is less than or equal to a preset maximum shooting threshold and greater than or equal to a preset minimum shooting threshold, subsequent operations are performed.

3. The method for identifying user intention of a scanning pen according to claim 1, characterized in that: The step of performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features includes: Perform scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates; The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; The scanning pen movement direction is calculated by the following formula: ； in, is the image movement direction of the scanning pen numbered i, is the vertical coordinate of the center of the i+1 scanning pen, is the vertical coordinate of the center of scanning pen No. i, is the horizontal coordinate of the center of the i+1 scanning pen, is the horizontal coordinate of the center of scanning pen No. i; The scanning pen movement angle is calculated by the following formula: ； in, is the image movement angle of the scanning pen numbered i; The scanning pen movement angle is calculated by the following formula: ； in, Indicates the image movement distance of scanning pen number i.

4. The method for identifying user intention of a scanning pen according to claim 3, characterized in that: The step of performing scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates includes: grayscale the numbered scanning pen image to obtain a scanning pen grayscale image; Performing contour detection on the grayscale image of the scanning pen to obtain a contour of the scanning pen; The minimum circumscribed circle is calculated for the scanning pen contour, and the center coordinates of the minimum circumscribed circle are used as the center coordinates of the scanning pen.

5. The method for identifying user intention of a scanning pen according to claim 1, characterized in that: The performing user intention analysis according to the scanning pen motion characteristics to obtain the user's real intention includes: The scanning pen movement characteristics include: scanning pen movement direction, scanning pen movement angle, and scanning pen movement distance; When the movement distance of the scanning pen is less than a preset distance threshold, the scanning pen image with a number corresponding to the movement distance of the scanning pen is determined to be an invalid image; When the movement distance of the scanning pen is greater than or equal to the preset distance threshold, the movement direction and the movement angle of the scanning pen are continuously judged, including: When the scanning pen movement angle corresponding to the scanning pen images with consecutive numbers exceeding the preset rotation number threshold is greater than the preset rotation angle threshold, it is determined that the user's real intention is "rotation"; When the scanning pen moves in a preset left-slide angle range, the user's real intention is determined to be "left-slide"; When the scanning pen moves in a preset right-slide angle range, it is determined that the user's real intention is "right-slide".

6. The method for identifying user intention of a scanning pen according to claim 5, characterized in that: After analyzing the user intention according to the scanning pen motion characteristics to obtain the user's real intention, the method further includes: When the user's real intention is "rotation", the corresponding function of the scanning pen is "mode switching"; When the user's real intention is "swipe left", the corresponding function of the scanning pen is "translate content"; When the user's actual intention is "swipe right", the corresponding function of the scanning pen is "read content aloud".

7. The method for identifying user intention of a scanning pen according to claim 1, characterized in that: After performing scanning pen motion analysis according to the numbered scanning pen image to obtain scanning pen motion features, the method further includes: Perform scanning pen center recognition on the numbered scanning pen image to obtain the scanning pen center coordinates; Perform interpolation processing according to the center coordinates of the scanning pen to obtain a motion trajectory curve; The motion trajectory of the scanning pen is drawn according to the motion trajectory curve.

8. A user intention recognition system for a scanning pen, characterized in that: include: The monitoring module is used to monitor whether the scanning pen is pressed. When the scanning pen is pressed, the camera is initialized to prepare for taking pictures. The shooting module is used to continuously shoot and generate pictures according to a preset frame difference while the scanning pen is pressed, and continuously read pictures and timestamps from the camera's cache; A numbering module, used to number the cached images taken out from the cache, add a timestamp to each of the cached images, and obtain a numbered scanning pen image; A motion analysis module, used for performing a scanning pen motion analysis according to the numbered scanning pen image to obtain a scanning pen motion feature; An intention analysis module, used to analyze the user's intention according to the scanning pen movement characteristics to obtain the user's real intention; The data uploading module is used to upload the image of the serial number scanning pen and the real intention of the user to the cloud server.

9. An electronic device, characterized in that: The invention comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for identifying user intention of the scanning pen according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for identifying user intention of the scanning pen according to any one of claims 1 to 7.

Images