Screen handwriting positioning method and device, electronic equipment and storage medium

Abstract

This invention discloses a screen handwriting positioning method, apparatus, electronic device, and storage medium. The method includes acquiring a handwriting image captured by a camera of a handwriting device in the handwriting direction, wherein the target screen in the handwriting direction displays at least one preset coded image; identifying a target coded image corresponding to the preset coded image in the handwriting image, and determining the target coordinates of the center of the handwriting image on the target screen based on the target coded image; and performing screen handwriting positioning on the handwriting device based on the target coordinates. This invention eliminates the need for a specific handwriting medium on the handwriting screen, enabling handwriting input even on ordinary non-touchscreen devices, thus improving the applicability of handwriting devices.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to a screen handwriting positioning method, device, electronic device, and storage medium. Background Technology

[0002] A screen stylus is an input tool that allows writing on touchscreen devices. It mimics the use of traditional writing tools (such as pencils or pens), allowing users to write, draw, or perform other interactive operations directly on the screen. Styluses play a significant role in improving the productivity, creativity, and convenience of mobile devices. However, currently, screen styluses require specific screens, such as capacitive, resistive, or infrared screens, and cannot perform handwriting input on ordinary non-touchscreen devices, thus limiting their applicability. Summary of the Invention

[0003] This invention provides a screen handwriting positioning method, device, electronic device, and storage medium, aiming to reduce the requirements for handwriting screens and improve the applicability of handwriting devices.

[0004] In a first aspect, embodiments of the present invention provide a screen handwriting positioning method, the method comprising:

[0005] Acquire a handwritten image captured by the camera of a handwriting device in the handwriting direction, wherein at least one preset encoded image is displayed on the target screen in the handwriting direction;

[0006] Identify the target encoded image corresponding to the preset encoded image in the handwritten image, and determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image;

[0007] The handwriting device is positioned on the screen based on the target coordinates.

[0008] Secondly, embodiments of the present invention provide a screen handwriting positioning device, the screen handwriting positioning device comprising:

[0009] The acquisition module is used to acquire a handwritten image captured by the camera of the handwriting device in the handwriting direction, wherein the target screen in the handwriting direction displays at least one preset coded image;

[0010] The recognition module is used to recognize the target encoded image corresponding to the preset encoded image in the handwritten image, and to determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image;

[0011] The positioning module is used to perform screen handwriting positioning of the handwriting device based on the target coordinates.

[0012] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the screen handwriting positioning methods provided in the embodiments of the present invention.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to perform the steps of any of the screen handwriting positioning methods provided in the embodiments of the present invention.

[0014] This invention first acquires a handwritten image captured by the camera of a handwriting device in the handwriting direction, and the target screen in the handwriting direction displays at least one preset coded image; it then identifies the target coded image corresponding to the preset coded image in the handwriting image, and determines the target coordinates of the center of the handwriting image on the target screen based on the target coded image; finally, it performs screen handwriting positioning of the handwriting device based on the target coordinates. This allows for screen handwriting positioning of the handwriting device even when there is no handwriting medium on the target screen, enabling handwriting input even on ordinary non-touchscreen devices, thus improving the applicability of handwriting devices. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating one embodiment of the screen handwriting positioning method provided in this invention.

[0017] Figure 2 This is a flowchart illustrating another embodiment of the screen handwriting positioning method provided in this invention.

[0018] Figure 3 A schematic diagram of a preset encoded image for a screen handwriting method provided in an embodiment of the present invention;

[0019] Figure 4 A schematic diagram of a preset encoding pattern on a positioning point in the screen handwriting method provided in an embodiment of the present invention;

[0020] Figure 5 A schematic diagram of encoding points for the screen handwriting method provided in an embodiment of the present invention;

[0021] Figure 6 This is a schematic diagram of a handwritten image of the screen handwriting method provided in an embodiment of the present invention;

[0022] Figure 7 A schematic diagram of a reference plane for the screen handwriting method provided in an embodiment of the present invention;

[0023] Figure 8 This is a schematic diagram of the screen handwriting positioning device provided in an embodiment of the present invention;

[0024] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0026] The following disclosure provides numerous different embodiments or examples to implement different features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and are not intended to be limiting. For example, in the following description, the formation of a first feature on or on a second feature may include embodiments where the first and second features are in direct contact, and may also include embodiments where additional features may be formed between the first and second features such that the first and second features may not be in direct contact. Furthermore, reference numerals and / or letters may be repeated in various instances of this disclosure. This repetition is for the purpose of simplicity and clarity and does not inherently define the relationship between the various embodiments and / or configurations discussed.

[0027] Furthermore, the use of terms such as "first," "second," etc., in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If a combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention. Moreover, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0028] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0031] As mentioned in the background section, the inventors discovered that current screen styluses require specific screens for use, such as capacitive, resistive, or infrared screens. This makes it impossible to achieve handwriting input on ordinary non-touchscreen devices, resulting in low applicability of screen-based handwriting devices.

[0032] This invention provides a screen handwriting positioning method, device, electronic device, and storage medium.

[0033] Specifically, the screen handwriting positioning method in this embodiment can be described from the perspective of the screen handwriting positioning device. The screen handwriting positioning device can be integrated into an electronic device. The electronic device can be the executing entity of the screen handwriting positioning method. The electronic device can be a handwriting device, such as a stylus or a handwriting finger sleeve. The electronic device can also be a screen. The electronic device can also be an intermediate device connected to the handwriting device and the screen respectively.

[0034] The following detailed description is provided in conjunction with the accompanying drawings. In this embodiment, an electronic device is used as the execution subject. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0035] To address the above problems, this invention discloses a screen handwriting positioning method. Please refer to [link / reference]. Figure 1 The specific process of this screen handwriting positioning method can be summarized in steps S10 to S30, wherein:

[0036] Step S10: Obtain the handwritten image captured by the camera of the handwriting device in the handwriting direction, and the target screen in the handwriting direction displays at least one preset coded image.

[0037] In this embodiment, the handwriting device can be a stylus, a handwriting ring, a handwriting finger sleeve, or other similar devices. This handwriting device can be used to implement handwriting functionality on any screen. When the handwriting device performs handwriting on a screen, the screen is positioned in the handwriting direction of the device, and this screen serves as the target screen, which is the screen where handwriting input needs to be performed via the handwriting device. The target screen has a display function and can display corresponding images, videos, interfaces, etc., according to control commands. To implement the handwriting function, the target screen needs to display at least one preset coded image. The target screen can be controlled to display at least one preset coded image through a handwriting program associated with it. Optionally, the handwriting program can be configured on the target screen to control the target screen and interact with the handwriting device.

[0038] The control command for displaying at least one preset coded image on the target screen can be triggered by the execution entity of the screen handwriting positioning in this embodiment, or by the user. In some embodiments, the target screen can be continuously controlled to display at least one preset coded image, or the target screen associated with the handwriting device can be controlled to display at least one preset coded image when screen handwriting positioning is required.

[0039] Optionally, prior to step S10, upon receiving a preset trigger signal, the target screen may be controlled to display at least one preset coded image. The preset trigger signal is a signal generated when a specific event occurs, indicating that screen handwriting positioning is currently required.

[0040] For example, a specific event could be the pen tip of a handwriting device pressing against the target screen. This specific event can be detected by a pressure sensor on the pen tip. When the trigger signal is received, handwriting mode is entered, requiring screen handwriting positioning. In addition, specific events could also include the user clicking the target screen, clicking the button or switch corresponding to the handwriting mode on the handwriting device, or launching the handwriting program on the target screen.

[0041] In this embodiment, the handwriting device is also equipped with a camera. This camera can be mounted facing the handwriting direction of the handwriting device, or its shooting direction can be adjusted according to instructions. This allows for the capture of handwriting images of the handwriting device in the handwriting direction. The handwriting direction refers to the direction of the pen tip on the handwriting device, such as the direction of the pen tip. When screen handwriting positioning is required, the handwriting device can be controlled to activate the camera to capture handwriting images in the handwriting direction. Subsequently, the camera can continuously capture handwriting images in the handwriting direction to obtain real-time handwriting images during the handwriting process, which are used for real-time screen handwriting positioning.

[0042] Understandably, the pen placement point can be the contact position between the handwriting device and the target screen. For example, if the handwriting device is a stylus, then the pen placement point can refer to the tip of the stylus. The pen placement point can also be a virtual contact position between the handwriting device and the target screen. For example, if the handwriting device is a handwriting ring, it may not actually contact the target screen, but rather a virtual contact position can be set between the handwriting device and the target screen.

[0043] In this embodiment, the target screen can display one or more preset coded images. Preferably, the target screen associated with the handwriting device can be controlled to display multiple preset coded images. By displaying multiple preset coded images on the target screen within the corresponding display range, the probability of the target coded image corresponding to the preset coded image appearing in the handwriting image can be increased. Furthermore, multiple target coded images can appear in the handwriting image, allowing for the comprehensive determination of target coordinates by combining multiple target coded images, thereby improving the accuracy and efficiency of screen handwriting positioning.

[0044] Step S20: Identify the target encoded image corresponding to the preset encoded image in the handwritten image, and determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image.

[0045] In this embodiment, a preset encoded image is displayed on the target screen. When handwriting is done on the target screen using a handwriting device, the target screen will appear in the handwriting direction of the handwriting device. The preset encoded image displayed on the target screen may be captured by the camera of the handwriting device, so that the handwritten image includes the target encoded image corresponding to the preset encoded image.

[0046] By identifying the target coded image contained in the handwritten image, and based on the image display position of the target coded image in the handwritten image and the display position information of the preset coded image on the target screen, the coordinate transformation relationship between the center of the handwritten image and the target screen can be determined, thereby determining the target coordinates of the center of the handwritten image on the target screen.

[0047] Step S30: Perform screen handwriting positioning for the handwriting device based on the target coordinates.

[0048] In this embodiment, the center of the handwritten image is the shooting center of the camera of the handwriting device, which is related to the position of the handwriting device. Therefore, the handwriting coordinates of the handwriting device on the target screen can be determined based on the target coordinates of the center of the handwritten image on the target screen. The handwriting coordinates refer to the coordinates of the pen placement point of the handwriting device on the target screen. Through the handwriting coordinates, the screen handwriting positioning of the handwriting device can be realized, and further, the movement trajectory of the handwriting device on the target screen can be tracked, thus realizing the handwriting function of the handwriting device on the target screen.

[0049] Optionally, step S30 may include: setting the target coordinates to the handwriting coordinates of the handwriting device on the target screen.

[0050] In some embodiments, the center of the handwritten image can be regarded as the pen landing point of the handwriting device. Then, the target coordinates can be set as the handwriting coordinates of the handwriting device on the target screen. Then, the pen strokes of the handwriting device can be directly generated according to the target coordinates to achieve screen handwriting positioning. Furthermore, the pen strokes can be displayed on the target screen according to the screen handwriting scenario to achieve the display of the handwriting movement trace of the handwriting device on the target screen.

[0051] Optionally, the camera can be set at the tip of the pen on the handwriting device, so that the center of the handwriting image is closer to the pen placement point of the handwriting device, and the target coordinates are closer to the actual handwriting coordinates. When the target coordinates are set as the handwriting coordinates of the handwriting device on the target screen, the accuracy of screen handwriting positioning can be improved.

[0052] Optionally, step S30 may include: acquiring handwriting posture information of the handwriting device, and determining the handwriting coordinates of the handwriting device on the target screen based on the handwriting posture information and the target coordinates.

[0053] In other embodiments, an inertial measurement unit or similar device may be provided on the handwriting device to acquire handwriting posture information of the handwriting device. Based on the handwriting posture information and the preset installation position of the camera on the handwriting device, the current relative position between the center of the handwriting image captured by the camera and the corresponding pen stroke point of the handwriting device can be determined. Then, based on the current relative position, the target coordinates of the center of the handwriting image on the target screen can be converted into the handwriting coordinates of the handwriting device on the target screen, thereby further improving the accuracy of screen handwriting positioning.

[0054] Optionally, during the handwriting process on the target screen using a handwriting device, the target screen is controlled to display a preset encoded image. The handwriting image captured by the camera of the handwriting device in the handwriting direction can be obtained in real time. Based on the target encoded image corresponding to the preset encoded image in the handwriting image, the center of the handwriting image is located on the target screen in real time to achieve real-time screen handwriting positioning of the pen placement point of the handwriting device, and further to track the movement trajectory of the pen placement point of the handwriting device on the target screen.

[0055] In the technical solution disclosed in this embodiment, a handwritten image in the handwriting direction captured by the camera of the handwriting device is acquired, and at least one preset coded image is displayed on the target screen in the handwriting direction; a target coded image corresponding to the preset coded image in the handwriting image is identified, and the target coordinates of the center of the handwriting image on the target screen are determined based on the target coded image; the handwriting device is then positioned on the screen based on the target coordinates. The screen handwriting positioning method provided in this embodiment has no other requirements for the target screen and can be adapted to any type of screen, including when there is no handwriting medium on the target screen, it can still perform screen handwriting positioning of the handwriting device, enabling ordinary non-touchscreen devices to achieve handwriting input, thereby improving the applicability of handwriting devices.

[0056] Optionally, step S10 includes:

[0057] The event image is captured by the camera of the handwriting device in the handwriting direction and set as the handwriting image, wherein the target screen flashes to display at least one preset encoded image.

[0058] In this embodiment, the camera on the handwriting device can capture event images. That is, the camera on the handwriting device has an event mode. This camera can be an event camera with only an event mode, or a fusion camera with both an event mode and an active programmable mode. The event mode is a working mode for imaging based on light intensity change events occurring on pixels. When a light intensity change event occurs on a pixel in event mode, light intensity change event information (including event occurrence time, event location, event type, etc.) is output for imaging. Compared to the active imaging mode, the event mode has many advantages such as high frame rate, high dynamic range, and low power consumption. Based on these advantages, event images in the handwriting direction can be acquired through the camera's event mode and set as the aforementioned handwriting image.

[0059] During the process of acquiring the event image, the target screen can be controlled to flash and display at least one preset coded image, so that the preset coded image has brightness variations on the target screen. For example, when a preset trigger signal to enter handwriting mode is received, the target screen is controlled to flash and display at least one preset coded image, and the camera set on the handwriting device is activated to capture an event image in the handwriting direction as a handwriting image, which is used for subsequent steps to recognize the target coded image corresponding to the preset coded image.

[0060] Since the handwriting images captured by the handwriting device are event images, if a preset encoded image is captured, the handwriting image will contain a target encoded image corresponding to the preset encoded image. Since the event image is based only on light intensity change events, most of the areas with pixel values ​​in the handwriting image belong to the target encoded image, which can improve the recognition and processing efficiency of the target encoded image and improve the screen handwriting response speed.

[0061] It is understandable that, when acquiring a handwriting image by capturing an event image in the handwriting direction using the camera of a handwriting device, it is preferable to control the target screen to flash and display multiple preset coded images. This increases the probability that the event image contains preset coded images, thereby achieving screen handwriting positioning. Furthermore, multiple target coded images can appear in the event image, and by combining these multiple target coded images, the target coordinates can be comprehensively determined, thereby improving the accuracy and efficiency of screen handwriting positioning.

[0062] Optionally, refer to Figure 2 Based on any of the above embodiments, in another embodiment of the screen handwriting positioning method of the present invention, the preset encoded image includes a positioning point and an encoded point located within the area of ​​the positioning point. The encoded point has encoded information, and the encoded information is generated based on the display position information of the corresponding preset encoded image on the target screen. Step S20 may include:

[0063] Step S21: Detect the handwritten image to determine the target positioning points of at least one target encoded image.

[0064] In this embodiment, the preset encoded image includes positioning points and encoding points. The encoding points are within the area where the positioning points are located. The area where the positioning points are located is an area determined based on the positioning points and a preset partitioning method, specifically, it can refer to the area enclosed by the positioning points. The range of the preset encoded image can be defined through the positioning points, and thus the range of the encoding points can also be defined. After determining the positioning points, the encoding points can be quickly determined within their respective areas. In the preset encoded image, the encoding points carry encoding information, which is generated based on the display position information of the preset encoded image on the target screen. Conversely, based on the encoding information on the encoding points, the display position information of the preset encoded image on the target screen can be deduced. The display position information refers to the display-related information of the preset encoded image, or the positioning points in the encoded image, on the target screen, to represent the display range, orientation, etc. of the preset encoded image on the screen.

[0065] In this embodiment, at least one preset encoded image can be controlled to be displayed on the target screen so that the camera can capture the target encoded image corresponding to the preset encoded image. It should be noted that in this embodiment, the shape, arrangement pattern of the preset encoded image, and the number and positions of the positioning points and encoding points in the preset encoded image are not limited.

[0066] For the convenience of image processing, in some embodiments, the shape of the preset encoded image can be square, and each preset encoded image includes five positioning points, which are respectively located at the four corners and the center of the preset encoded image, for more accurate positioning of the area of the preset encoded image.

[0067] Exemplarily, the shape of the preset encoded image can be as Figure 3 shown. In this embodiment, the preset encoded image is denoted as Block, and each grid in Block is denoted as Cell. As Figure 3 shown, the preset encoded image is square, where C represents the positioning grid, and I_X, I_Y, I_Z, I_1, I_0 are encoding points. Among them, I_X is the encoding of the X coordinate of the display area of Block on the target screen, I_Y is the encoding of the Y coordinate of the display area of Block on the target screen, I_Z is the check code of all I_X and I_Y, and I_1, I_0 are used for positioning the direction.

[0068] In this embodiment, the encoding patterns on the positioning points and encoding points in the preset encoded image are different. For example, the encoding image of the positioning point can be Figure 4 the "square" shape as shown, and the encoding pattern of the encoding point can be as Figure 5The point encoding is shown. In this embodiment, the area occupied by the positioning point and the encoded pattern on the encoding point in the preset encoded image is not limited; the areas of the two can be the same or different.

[0069] When the camera of the handwriting device can capture at least one preset encoded image, by detecting the handwriting image captured by the camera of the handwriting device, a target positioning point in the target encoded image corresponding to at least one preset encoded image can be determined in the handwriting image. It can be understood that the target positioning point corresponds to the positioning point in the preset encoded image corresponding to the target encoded image.

[0070] Step S22: For each target encoded image, based on the target localization points of the target encoded image, determine the target encoding points of the target encoded image in the handwritten image.

[0071] In this embodiment, after detecting the handwritten image, at least one target localization point can be determined in the target encoded image. Subsequent processing is required on a per-target encoded image basis. For each target encoded image in the at least one target encoded image, the target localization point of that target encoded image is used to determine the target encoded point of that target encoded image. It is understood that the target encoded point corresponds to the encoded point in the preset encoded image corresponding to the target encoded image it belongs to.

[0072] Specifically, for the same preset encoded image, if the encoded point is located within the area of ​​the corresponding positioning point, the positioning point limits the range of the preset encoded image and also the range of the encoded point. The preset encoded image corresponds to the target encoded image. Therefore, after determining the target positioning point of a target encoded image in a handwritten image, the target encoded point in the handwritten image can be quickly determined based on the area where the target positioning point is located. The area where the target positioning point is located refers to the area determined based on the target positioning point and the aforementioned preset division method (the preset division method corresponding to the area where the positioning point is located in the preset encoded image corresponding to the target positioning point). For example, if the area where the positioning point is located is the area enclosed by the positioning points, then the area where the target positioning point is located is also the area enclosed by the target positioning points.

[0073] It is understood that the target positioning point and target encoding point determined in this embodiment belong to the same target encoding image, and correspond to the positioning point and encoding point in the preset encoding image corresponding to this target encoding image, respectively.

[0074] Step S23: Determine the target coordinates based on the target coding information and target positioning point corresponding to at least one target coding image at the target coding point.

[0075] In this embodiment, after determining the target positioning points of at least one target coded image, the target encoding points of the target preset coded image can be determined based on these target positioning points. Thus, the target positioning points and target encoding points corresponding to at least one target coded image can be obtained. For each target coded image, its corresponding target positioning points and target encoding points correspond to its corresponding preset coded image. Since the encoding points of the preset coded image have encoding information generated based on the display position information of the preset coded image on the target screen, and the target encoding points correspond to the encoding points, in the handwritten image, the target encoding points also have target encoding information. By identifying the target encoding information on the target encoding points, the target display position information can be obtained. This target display position information is generally the display position information of the preset coded image corresponding to this target coded image on the target screen. Therefore, even when multiple preset coded images are displayed on the target screen, this method can quickly and accurately determine the display position information of the preset coded image corresponding to each target coded image on the target screen.

[0076] Based on the target positioning points, the image position information of the target coded image on the handwritten image can be determined. This image position information includes the image coordinates and orientation information of the target coded image on the handwritten image. Based on the target encoding information at the target encoding points, the target display position information can be determined, which serves as the display position information of its corresponding preset coded image on the target screen. Based on the correspondence between the target coded image and the preset coded image, the coordinates of the center of the handwritten image on the target screen can be calculated using both the image position information and the target display position information. It can be understood that the target positioning points and target encoding points corresponding to each target coded image can be used to determine the coordinates of the center of the handwritten image on the target screen.

[0077] In some embodiments, if only one target positioning point and target encoding point corresponding to a target encoded image are identified, the coordinates of the center of the handwritten image on the target screen, determined based on the target encoded image, can be set as the target coordinates of the center of the handwritten image on the target screen.

[0078] In some embodiments, if target positioning points and target encoding points corresponding to multiple target encoded images are identified, the coordinates of the center of the handwritten image on the target screen can be determined based on any one of the target encoded images, and these coordinates can be set as target coordinates. Alternatively, the coordinates of the centers of multiple handwritten images on the target screen can be determined based on multiple target encoded images, and the target coordinates can be set comprehensively based on these coordinates.

[0079] In some alternative implementations, if multiple target positioning points and target coding points corresponding to target coded images are identified, the target display position information corresponding to the target coded image is determined based on the target coding information on the target coding point corresponding to each target coded image. Based on the target positioning point corresponding to each target coded image, the image position information of the target coded image on the handwritten image can be determined, and multiple sets of associated target display position information and image position information can be obtained. Then, based on at least two sets of associated target display position information and image position information, the coordinate transformation relationship between the target screen and the handwritten image is determined, and the center coordinates of the handwritten image are transformed to the target screen to obtain the target coordinates.

[0080] Optionally, the positioning point has a preset coding pattern, and the handwritten image is detected to determine the target positioning point in the handwritten image corresponding to the positioning point, including:

[0081] Based on a preset coding pattern, the correlation score between pixels and positioning points in a handwritten image is calculated.

[0082] Based on the correlation scores of each pixel in the handwritten image, at least one target localization point in the target encoded image is determined.

[0083] In this embodiment, the encoding patterns on the positioning points and encoding points in the preset encoded image can be different. The positioning points have the preset encoded image. Based on the comparison between the preset encoded pattern and various regions of the handwritten image, the correlation score between each pixel in the handwritten image and the positioning point in the preset encoded image can be obtained. The higher the correlation score, the greater the probability that the pixel is the target positioning point corresponding to the positioning point in the preset encoded image. Based on the correlation score of each pixel in the handwritten image, pixels in the handwritten image that may be target positioning points can be identified, and thus at least one target positioning point in the target encoded image can be identified.

[0084] By using a preset coding pattern for the positioning points, the correlation score between each pixel on the handwritten image and the positioning points is calculated, thereby enabling the target positioning points to be identified quickly and accurately from the handwritten images, improving the accuracy and efficiency of screen handwriting positioning.

[0085] Optionally, based on a preset encoding pattern, the correlation score between pixels and positioning points in the handwritten image is calculated, including:

[0086] Based on a preset coding pattern, a sliding window operation is performed on the handwritten image to calculate the Hamming distance between the sub-window image corresponding to each pixel and the preset coding pattern.

[0087] The correlation score of the corresponding pixel is determined based on the Hamming distance.

[0088] In this embodiment, refer to Figure 6 A 15x15 handwritten image I1 is shown. When the handwritten image is an event image, in I1, 1 represents a positive event, -1 represents a negative event, and 0 represents no event. A corresponding template image I2 is set based on a preset encoding pattern. The size of I2 can be set according to the actual situation. A sliding window operation is performed on the handwritten image I1 based on the template image I2. A sub-window can be divided in the handwritten image I1 with each pixel as the center. Then, the Hamming distance between the sub-window image and the template image I2, that is, the Hamming distance between the sub-window image and the preset encoding pattern, can be calculated. Hamming distance is used to compare the degree of difference between two images and can be used to characterize the correlation score between a pixel and a location point in the preset encoding image. The larger the Hamming distance, the larger the correlation score of the corresponding pixel. The Hamming distance corresponding to each pixel can be divided by the size of the sub-window for normalization processing to obtain the correlation score of the corresponding pixel for subsequent calculations.

[0089] By performing a sliding window operation on the handwritten image based on a preset coding pattern, and calculating the Hamming distance between the sub-window image corresponding to each pixel and the preset coding pattern, the correlation score of each pixel can be obtained quickly and accurately, further improving the accuracy and efficiency of screen handwriting positioning.

[0090] Optionally, the preset coded image includes a first positioning point and a preset number of second positioning points, the first positioning point being located at a preset position, and the preset position being determined based on the second positioning points and a preset geometric relationship;

[0091] Based on the correlation scores of each pixel in the handwritten image, at least one target localization point in the target encoded image is determined, including:

[0092] The first set of pixels is selected based on the correlation scores of each pixel in the handwritten image;

[0093] For each target pixel in the first pixel set, a preset number of first pixels are selected from the first preset range corresponding to the target pixel, and the first pixels belong to the first pixel set.

[0094] Based on the first pixel and the preset geometric relationship, determine the target position and calculate the target position and the relative distance to the target pixel.

[0095] If the relative distance is less than the preset distance, the target pixel and the first pixel are set as the target positioning point of a target encoded image.

[0096] In this embodiment, each preset coded image includes a first positioning point and a preset number of second positioning points. The first positioning point is located at a preset position, which is determined based on the second positioning point and a preset geometric relationship. The preset geometric relationship can characterize the relative positions and relationships between the preset number of points and a preset. The preset geometric relationship can be represented by geometric information such as distance, angle, parallelism, perpendicularity, and intersection, so that the positional relationship between the first positioning point and the second positioning point can be calculated.

[0097] It is understood that the preset number is a positive integer. In some embodiments, the preset number is greater than or equal to 2, such that the number of the first positioning point plus the number of the second positioning point is at least greater than or equal to 3. That is, the preset image contains at least 3 positioning points, so that the positioning points in the preset image can form a unique area, limiting the range of the coding point, thereby simplifying the operation of determining the area where the positioning point is located or determining the area where the target positioning point is located.

[0098] Based on the correlation scores of each pixel in the handwritten image, a set of pixels with high correlation scores can be selected from a number of pixels in the handwritten image to form the first pixel set. The selection method is not limited here.

[0099] Each pixel in the first pixel set can be considered a target pixel. A target pixel belonging to the first pixel set indicates that it may belong to the target encoded image, and further evaluation is needed to determine if it corresponds to the aforementioned first positioning point. For each target pixel in the first pixel set, a predetermined number of first pixels belonging to the first pixel set are selected from the first preset range corresponding to the target pixel. This first preset range can be set according to actual conditions, such as the size of the preset image or the preset position of the first positioning point in the preset image. It should be noted that the first pixels belong to all pixels in the first pixel set except for the target pixels.

[0100] For example, if there is one first positioning point and three second positioning points, then for each target pixel in the first pixel set, three first pixels that also belong to the first pixel set need to be randomly selected from its corresponding first preset range. These three first pixels do not include the target pixel.

[0101] Based on the selected preset number of first pixels, the corresponding target position can be determined through the aforementioned preset geometric relationship, and the relative distance between the target position and the target pixel can be calculated. If the relative distance is less than the preset distance, it indicates that the geometric relationship between the target pixel and these first pixels is consistent with the geometric relationship of the preset encoded image, and they correspond to the same preset encoded image, all belonging to the same target encoded image. Specifically, the target pixel corresponds to the first positioning point of this preset encoded image, and the first pixel corresponds to the second positioning point of this preset encoded image. The target pixel and these first pixels can all be set as target positioning points of the same target preset encoded image.

[0102] In other words, each target pixel and its corresponding first pixel can be bound together and classified into a group of target positioning points. This group of target positioning points belongs to the same target encoded image and comes from the same preset encoded image. The corresponding target encoded point can be determined by using this group of target positioning points as a whole to define the range. Thus, at least one target encoded point and target positioning point of the target encoded image can be obtained for screen handwriting positioning.

[0103] Optionally, the preset encoded image includes five positioning points, one of which is located at a preset position that is the midpoint of the diagonal determined based on the other four positioning points.

[0104] In this embodiment, a specific design scheme for a preset encoded image is also provided. The preset encoded image includes five positioning points, one of which is located at a preset position. This preset position is determined based on the other four positioning points and a preset geometric relationship, which refers to the intersection of the diagonals of the four positioning points. This ensures that the preset position of one positioning point is at the intersection of the diagonals of the other four positioning points, and this preset position is also the center position relative to the other positioning points. Through this specific design scheme for the preset encoded image, the range of the preset encoded image can be more accurately defined, and the recognition rate of the target encoded image corresponding to the preset encoded image in handwritten images can be improved.

[0105] Optionally, a first set of pixels is selected based on the correlation scores of each pixel in the handwritten image, including:

[0106] For each pixel in the handwritten image, if the correlation score of the pixel is greater than the preset score, the pixel is recorded in the second pixel set.

[0107] If the second pixel in the second pixel set has the highest correlation score within its corresponding second preset range, then the second pixel is recorded in the first pixel set to filter out the first pixel set.

[0108] In this embodiment, based on the above-mentioned preset encoded image design, after scoring the correlation of each pixel on the handwritten image, the following scheme for filtering the first pixel set can be adopted to improve the accuracy and efficiency of identifying target positioning points.

[0109] Specifically, after determining the correlation score between each pixel in the handwritten image and the location point of the preset encoded image, each pixel is first preliminarily screened to reduce subsequent computation and improve accuracy. First, pixels in the handwritten image with correlation scores greater than the preset score are identified and recorded in a second pixel set. The preset score can be set according to actual conditions (such as the total number of points in the template image I2 mentioned above). Then, for each second pixel in the second pixel set, a second preset range corresponding to the second pixel is selected in the handwritten image. The second preset range is set according to actual conditions. If the second pixel is still the pixel with the highest correlation score within the second preset range, then it can be recorded in the first pixel set. After iterating through each second pixel in the second pixel set, the final first pixel set is obtained, which is used for subsequent identification of at least one target location point corresponding to the target encoded image.

[0110] To better understand, based on the above embodiments, a specific implementation plan is provided below:

[0111] refer to Figure 3 The preset encoded image includes five positioning points C, one of which is located at a preset position determined by the midpoint of the diagonal based on the other four positioning points C. After determining the correlation score between each pixel in the handwritten image and the positioning points in the preset encoded image, each pixel is initially screened: for each pixel in the handwritten image, if the correlation score of the pixel is greater than the preset score, the pixel is recorded in the second pixel set L2.

[0112] Local maximum point detection is performed based on the second pixel set L2: For each second pixel in the second pixel set L2, a second preset range S2 corresponding to the second pixel is selected in the handwritten image. The second preset range S2 is set according to the actual situation. If the second pixel is still the pixel with the highest correlation score within the second preset range S2, then the second pixel is recorded in the first pixel set L1.

[0113] Based on the first pixel set L1, the positioning of the corresponding positioning point group of the preset encoded image is performed: Target pixels are selected from the first pixel set L1, where some or all pixels can be used as target pixels. For each target pixel L1_tmp in the first pixel set L1, a first preset range [S2, S3] corresponding to the target pixel L1_tmp is selected in the handwritten image. The first preset range [S2, S3] can be set according to the size of the preset encoded image, the camera shooting height, and other actual conditions, so that the range of the first preset range [S2, S3] includes other target positioning points as accurately as possible. Obtain other pixels within the first preset range [S2, S3] to form a pixel set L1_C. Select four first pixels from the pixel set L1_C and calculate the relative distance between the intersection of the diagonals of these four first pixels and the target pixel L1_tmp. If the relative distance is less than a preset distance, then these target pixel L1_tmp and these four first pixels can be used as a positioning point group G_tmp, and this positioning point group G_tmp is recorded in the third pixel set L3. Each positioning point group G_tmp can determine the region of the target encoded image corresponding to a preset encoded image within the handwritten image.

[0114] After initial screening based on the correlation scores of each pixel in the handwritten image, further screening is performed based on the local maximum correlation score. Then, through geometric positioning, at least one set of target positioning points corresponding to a preset image can be quickly and accurately determined, thereby further improving the accuracy and efficiency of screen handwriting positioning.

[0115] Optionally, step S22 may include:

[0116] Based on the camera's intrinsic parameters, calculate the first homography matrix from the region where the target localization point of at least one target-coded image is located to the reference plane.

[0117] Obtain the coordinates of the reference coded point recorded on the reference plane;

[0118] The target coding point is determined based on the first homography matrix and the coordinates of the reference coding point.

[0119] In this embodiment, the target screen can display at least one preset coded image. Therefore, it is necessary to obtain at least one target positioning point of the target coded image. These target positioning points belong to the target coded image corresponding to the preset coded image. The area where the target positioning point of the same preset coded image is located can determine the area where the target coded image is located. The area where the target positioning point is located can be divided according to the area where the positioning point is located in the corresponding preset coded image. Specifically, it can refer to the area enclosed by the target positioning point.

[0120] Based on the target location point of at least one target encoded image and the intrinsic parameters of the camera on the handwriting device, the first homography matrix H1 from the region where the target location point is located to the reference plane can be calculated, or in other words, the first homography matrix H1 between the reference plane and the plane where the target encoded image is located.

[0121] It is understandable that the regions where the target positioning point is located in the reference plane, the target screen, and the handwritten image do not belong to the same planar coordinate system. Therefore, it is necessary to realize the transformation of the planar coordinate system by calculating the homography matrix of the plane.

[0122] The reference plane includes reference positioning points and reference coding points, which correspond to the positioning points and coding points in the preset coding image, and to the target positioning points and target coding points in the target coding image, respectively. The positions of the reference positioning points and reference coding points in the reference plane are known, that is, the coordinates of the reference positioning points and reference coding points on the reference plane are known.

[0123] Each preset encoded image can be identical, meaning the relative positions between its positioning points and encoding points can be the same. Therefore, a set of universal reference positioning point coordinates and reference encoding point coordinates can be recorded on the reference plane. The universal reference encoding point coordinates recorded on the reference plane can then be obtained. Based on the first homography matrix H1 and the reference encoding point coordinates, the coordinates of the target encoding point within the region where the target positioning point of at least one target encoded image is located are calculated on the handwritten image, thereby identifying the target encoding point from the handwritten image.

[0124] For example, Figure 7 A reference plane and the coordinates of its points are shown. The position coordinates of each point on the cell are known on the reference plane. Based on the first homography matrix H1, the position coordinates of the cells on the reference plane can be mapped onto the event image l1, thus obtaining the position coordinates of each cell on event image l1, and consequently, the encoded information points corresponding to each set of target encoded images. The coordinates of each point on the reference plane can be set according to the arrangement and order of each cell in the preset encoded image.

[0125] By calculating the first homography matrix from the target location area to the reference plane using the camera's intrinsic parameters, and transforming the coordinates of the reference coding point on the reference plane, the target coding point on the handwritten image can be quickly and accurately identified, further improving the accuracy and efficiency of screen handwriting positioning.

[0126] Optionally, based on the target coding information and target positioning point at the target coding point, the target coordinates are determined, including:

[0127] For each target encoded image, target display position information is obtained by processing the target encoding information corresponding to the target encoded image. Based on the target display position information and the target positioning point of the target encoded image, the second homography matrix from the region where the target positioning point is located to the target screen is calculated.

[0128] The target coordinates are determined based on the second homography matrix corresponding to at least one target encoded image.

[0129] In this embodiment, for each target encoded image, the encoded point has encoded information generated based on the display position information of the preset encoded image on the target screen. The target encoded point of the handwritten image corresponds to the encoded point and also contains target encoded information corresponding to the encoded information. Based on the position of the target encoded point, the target encoded information is extracted from the handwritten image and processed to obtain the target display position information. This target display position information can be used as the display position information of the target positioning point and the preset encoded pattern corresponding to the target encoded point on the target screen. According to the target display position information, the coordinates of the positioning point corresponding to the target positioning point on the target screen can be determined. According to the coordinates and the coordinates of the target positioning point on the handwritten image, the second homography matrix H2 between the area where the target positioning point is located and the target screen can be calculated. Each target encoded image can be corresponding to a second homography matrix H2. According to the second homography matrix H2 corresponding to at least one target encoded image, the coordinate transformation relationship between the handwritten image and the target plane can be determined. Then, the coordinates of the center of the handwritten image can be projected onto the target screen, thereby determining the target coordinates of the center of the handwritten image on the target screen.

[0130] By using at least one target encoded image, calculating the second homography matrix between the region where at least one target positioning point is located and the target screen, the target coordinates of the center of the handwritten image on the target screen can be quickly and accurately determined, which can further improve the accuracy and efficiency of screen handwriting positioning.

[0131] Optionally, the encoded information includes a check code, and after step S22, it further includes:

[0132] Perform non-empty code verification on the target encoded information and obtain the target check code in the target encoded information that has passed the non-empty code verification;

[0133] Based on the target check code, the validity of the target encoded information for non-empty code verification is checked.

[0134] Based on the target encoding information that has passed the validity check, the target coordinates are determined by analyzing the target encoding information and target positioning points corresponding to at least one target encoding image.

[0135] In this embodiment, as Figure 3As shown, the encoding information at the encoding points in the preset encoded image includes not only encoding information about the display position but also a check code. Since the target encoding points in the handwritten image are determined based on the target positioning points, the target encoding points and the target positioning points correspond to each other and both belong to the same preset encoded image. The target encoding information at the target encoding points also corresponds to the encoding information and includes a target check code.

[0136] To improve the accuracy of identifying target encoded images, it is also necessary to perform validity checks on the target encoded information corresponding to each target encoded image. Before performing validity checks, a non-empty code check is performed on the target encoded information at each target encoded point. After the non-empty code check passes, the target check code in the target encoded information that passed the non-empty code check is obtained. Based on the target check code that passed the non-empty code check, the validity checks of other target encoded information that passed the non-empty code check are performed. If other target encoding information that passes the non-empty code check passes the validity check, then the target encoding information is determined to be valid encoding information. The corresponding target encoding image, its target positioning point, and target encoding point are all valid. Then, the target encoding information and target positioning point on the target encoding point in the target encoding image can be used to determine the above-mentioned target coordinates. The target coordinates can be determined based on the target encoding information and target positioning point on the target encoding point corresponding to at least one target encoding image. If the validity check fails, then the corresponding target encoding image, its target positioning point, and target encoding point are all invalid. Then, the target encoding information and target positioning point in the target encoding image cannot be used for screen handwriting positioning. It is necessary to ignore or cancel the relevant information of this target encoding image, including the target positioning point and target encoding point and their target encoding information within the target encoding image.

[0137] In some embodiments, if the target encoded image corresponding to the preset encoded image is not identified, or if a valid target encoded image is not identified, the process can return to step S10 to obtain the handwritten image captured by the camera of the handwriting device in the handwriting direction. Furthermore, the target screen can be controlled to adjust and display the preset encoded image, and after adjustment, step S10 can be re-executed.

[0138] Furthermore, if the target encoded image corresponding to the preset encoded image is not identified within the preset time, or if a valid target encoded image is not identified, the process can return to step S10 to obtain the handwritten image captured by the camera of the handwriting device in the handwriting direction. The process can also further control the target screen to adjust and display the preset encoded image, and after adjustment, step S10 can be re-executed.

[0139] Furthermore, after step S20, the following steps are also included:

[0140] Update the display range of the preset encoded image on the target screen based on the target coordinates;

[0141] The target screen is controlled to adjust the display position of the preset encoded image according to the display range, and then returns to the step of acquiring the handwritten image in the handwriting direction captured by the camera of the handwriting device.

[0142] In this embodiment, after determining the target coordinates of the center of the handwritten image on the target screen in real time, the target coordinates can be used as the center to update the display range of the preset encoded image on the target screen. This display range can be a preset range divided with the target coordinates as the center. The target screen is controlled to adjust the display of the preset encoded image according to the updated display range. Specifically, the display position and number of preset encoded images can be adjusted so that the preset encoded images displayed on the target screen are all within the display range after adjustment.

[0143] Optionally, the area of ​​the display range can be smaller than the size of the target screen, so that the preset encoding pattern is only displayed near the handwriting device, avoiding full-screen display of the preset encoding pattern, which can improve the screen viewing experience and screen display effect in handwriting mode.

[0144] Optionally, the aforementioned preset range can be smaller than the camera's shooting range, so that the display range determined by the target coordinates does not exceed the camera's shooting range. In this way, the preset encoded image displayed within the display range can be more easily captured by the handwriting device's camera, which can improve the efficiency of screen handwriting positioning and further limit the display range of the preset encoded image, thereby further improving the screen's visual appeal and display effect in handwriting mode.

[0145] In this way, during the real-time screen handwriting positioning process of the handwriting device, the display of the preset coded image on the target screen can be adjusted in real time according to the movement of the handwriting device. This can improve the screen viewing experience and display effect in handwriting mode, and also increase the probability that the handwriting device can capture the preset coded image, thereby further improving the accuracy and efficiency of screen handwriting positioning.

[0146] Optionally, step S20 includes:

[0147] Query the display position information of the preset encoded image corresponding to the target encoded image on the target screen;

[0148] The target coordinates are determined based on the image position information and display position information of the target encoded image in the handwritten image.

[0149] In this embodiment, the execution subject of the screen handwriting positioning method can communicate with the display control unit of the target screen regarding the preset coded image, and directly query the display position information corresponding to at least one preset coded image displayed on the target screen.

[0150] In some embodiments, if only one preset encoded image is displayed on the target screen, then at most one target encoded image appears in the handwritten image. Then, the image position information of this target encoded image can be matched with the unique display position information found to determine the coordinate transformation relationship between the handwritten image and the target screen, and then the target coordinates of the center of the handwritten image on the target screen can be determined.

[0151] In some embodiments, if multiple preset encoded images are displayed on the target screen, and at least one target encoded image appears in the handwritten image, the patterns of the preset encoded images may be different. For each target encoded image in the handwritten image, image matching can be performed with each preset encoded image displayed on the target screen to determine the preset encoded image corresponding to the target encoded image. The display position information of the corresponding preset encoded image is retrieved. Based on the image position information of at least one target encoded image in the handwritten image and the display position information corresponding to this target encoded image, the coordinate transformation relationship between the handwritten image and the target screen is determined, thereby determining the target coordinates of the center of the handwritten image on the target screen.

[0152] By using communication connections and image matching, the location information of the target coded image and the display position information of its corresponding preset coded image on the target screen can be quickly determined, thereby improving the efficiency of screen handwriting positioning.

[0153] This embodiment also provides a screen handwriting positioning device, which can be integrated into an electronic device. For example... Figure 8 As shown, the screen handwriting positioning device may include:

[0154] The acquisition module 1001 is used to acquire a handwritten image in the handwriting direction captured by the camera of the handwriting device, and the target screen in the handwriting direction displays at least one preset coded image.

[0155] The recognition module 1002 is used to recognize the target encoded image corresponding to the preset encoded image in the handwritten image, and to determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image;

[0156] The positioning module 1003 is used for screen handwriting positioning of the handwriting device based on the target coordinates.

[0157] Optionally, the acquisition module 1001 is also used for:

[0158] The event image is captured by the camera of the handwriting device in the handwriting direction and set as the handwriting image, wherein the target screen flashes to display at least one preset encoded image.

[0159] Optionally, the preset coded image includes a positioning point and coded points within the area where the positioning point is located. The coded points have coded information, which is generated based on the display position information of the corresponding preset coded image on the target screen. The recognition module 1002 is further used for:

[0160] Detect handwritten images to determine at least one target localization point in the target encoded image;

[0161] For each target encoded image, based on the target localization points of the target encoded image, determine the target encoded points of the target encoded image in the handwritten image;

[0162] The target coordinates are determined based on the target coding information and target positioning point corresponding to at least one target coding image.

[0163] Optionally, the positioning point has a preset coded pattern; the recognition module 1002 is also used for:

[0164] Based on a preset coding pattern, the correlation score between pixels and positioning points in a handwritten image is calculated.

[0165] Based on the correlation scores of each pixel in the handwritten image, at least one target localization point in the target encoded image is determined.

[0166] Optionally, the identification module 1002 is also used for:

[0167] Based on a preset coding pattern, a sliding window operation is performed on the handwritten image to calculate the Hamming distance between the sub-window image corresponding to each pixel and the preset coding pattern.

[0168] The correlation score of the corresponding pixel is determined based on the Hamming distance.

[0169] Optionally, the preset coded image includes a first positioning point and a preset number of second positioning points, the first positioning point being located at a preset position, the preset position being determined based on the second positioning points and a preset geometric relationship; the recognition module 1002 is further configured to:

[0170] The first set of pixels is selected based on the correlation scores of each pixel in the handwritten image;

[0171] For each target pixel in the first pixel set, a preset number of first pixels are selected from the first preset range corresponding to the target pixel, and the first pixels belong to the first pixel set.

[0172] Based on the first pixel and the preset geometric relationship, determine the target position and calculate the target position and the relative distance to the target pixel.

[0173] If the relative distance is less than the preset distance, the target pixel and the first pixel are set as the target positioning point of a target encoded image.

[0174] Optionally, the preset encoded image includes five positioning points, one of which is located at a preset position that is the midpoint of the diagonal determined based on the other four positioning points.

[0175] Optionally, the identification module 1002 is also used for:

[0176] For each pixel in the handwritten image, if the correlation score of the pixel is greater than the preset score, the pixel is recorded in the second pixel set.

[0177] If the second pixel in the second pixel set has the highest correlation score within its corresponding second preset range, then the second pixel is recorded in the first pixel set to obtain the first pixel set.

[0178] Optionally, the identification module 1002 is also used for:

[0179] Based on the camera's intrinsic parameters, calculate the first homography matrix from the region where the target localization point of at least one target-coded image is located to the reference plane.

[0180] Obtain the coordinates of the reference coded point recorded on the reference plane;

[0181] The target coding point is determined based on the first homography matrix and the coordinates of the reference coding point.

[0182] Optionally, the identification module 1002 is also used for:

[0183] For each target encoded image, target display position information is obtained by processing the target encoding information corresponding to the target encoded image. Based on the target display position information and the target positioning point of the target encoded image, the second homography matrix from the region where the target positioning point is located to the target screen is calculated.

[0184] The target coordinates are determined based on the second homography matrix corresponding to at least one target encoded image.

[0185] Optionally, the identification module 1002 is also used for:

[0186] Perform non-empty code verification on the target encoded information and obtain the target check code in the target encoded information that has passed the non-empty code verification;

[0187] Based on the target check code, the validity of the target encoded information for non-empty code verification is checked.

[0188] Based on the target encoding information that has passed the validity check, the target coordinates are determined by analyzing the target encoding information and target positioning points corresponding to at least one target encoding image.

[0189] Optionally, the identification module 1002 is also used for:

[0190] Perform non-empty code verification on the target encoded information and obtain the target check code in the target encoded information that has passed the non-empty code verification;

[0191] Based on the target check code, the validity of the target encoded information for non-empty code verification is checked.

[0192] Based on the target encoding information that has passed the validity check, the target coordinates are determined by analyzing the target encoding information and target positioning points corresponding to at least one target encoding image.

[0193] Optionally, the screen handwriting positioning device also includes a control module, which is further used for:

[0194] Update the display range of the preset encoded image on the target screen based on the target coordinates;

[0195] The target screen is controlled to adjust the display of the preset encoded image according to the changed display range, and then returns to the step of obtaining the handwriting image captured by the camera of the handwriting device in the handwriting direction.

[0196] Optionally, the positioning module 1003 is also used for:

[0197] Set the target coordinates to the handwriting coordinates of the handwriting device on the target screen.

[0198] Optionally, the positioning module 1003 is also used for:

[0199] Obtain handwriting posture information from the handwriting device;

[0200] Based on the handwriting posture information and target coordinates, determine the handwriting coordinates of the handwriting device on the target screen.

[0201] In this embodiment, a handwritten image captured by the camera of the handwriting device in the handwriting direction is first acquired. The target screen in the handwriting direction displays at least one preset coded image. A target coded image corresponding to the preset coded image in the handwriting image is identified, and the target coordinates of the center of the handwriting image on the target screen are determined based on the target coded image. The handwriting device is then positioned on the screen based on these target coordinates. This allows for screen handwriting positioning even when there is no handwriting medium on the target screen, enabling handwriting input even on ordinary non-touchscreen devices and improving the applicability of the handwriting device.

[0202] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0203] like Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 1100 includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0204] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a CPU, GPU, network processor (NP), etc., and can implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this invention.

[0205] In this embodiment of the invention, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as:

[0206] Acquire a handwritten image captured by the camera of the handwriting device in the handwriting direction, and the target screen in the handwriting direction displays at least one preset encoded image;

[0207] In a handwritten image, a target encoded image corresponding to a preset encoded image is identified, and the center of the handwritten image on the target screen is determined based on the target encoded image.

[0208] Screen handwriting positioning for handwriting devices based on target coordinates.

[0209] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0210] Optional, such as Figure 9As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0211] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be used as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0212] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0213] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0214] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0215] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1107 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0216] although Figure 9 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity unit, Bluetooth unit, etc., which will not be described in detail here.

[0217] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0218] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0219] To this end, embodiments of the present invention provide a computer-readable storage medium storing a plurality of computer programs, which can be loaded by a processor to execute any of the screen handwriting positioning methods provided in the embodiments of the present invention. The computer program can execute the following steps of the screen handwriting positioning method:

[0220] Acquire a handwritten image captured by the camera of the handwriting device in the handwriting direction, and the target screen in the handwriting direction displays at least one preset encoded image;

[0221] In a handwritten image, a target encoded image corresponding to a preset encoded image is identified, and the center of the handwritten image on the target screen is determined based on the target encoded image.

[0222] Screen handwriting positioning for handwriting devices based on target coordinates.

[0223] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0224] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0225] Since the computer program stored in the computer-readable storage medium can execute any of the screen handwriting positioning methods provided in the embodiments of the present invention, it can achieve the beneficial effects that any of the screen handwriting positioning methods provided in the embodiments of the present invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0226] In the above embodiments of the screen handwriting positioning device, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the screen handwriting positioning device, computer-readable storage medium, computer program product, electronic device, and their corresponding units described above can be referred to the description of the screen handwriting positioning method in the above embodiments, and will not be repeated here.

[0227] The foregoing has provided a detailed description of a screen handwriting positioning method, screen handwriting positioning device, electronic device, computer-readable storage medium, and computer program product provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A screen handwriting positioning method, characterized in that, The screen handwriting positioning includes: Acquire a handwritten image captured by the camera of a handwriting device in the handwriting direction, wherein at least one preset encoded image is displayed on the target screen in the handwriting direction; Identify the target encoded image corresponding to the preset encoded image in the handwritten image, and determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image; The handwriting device is positioned on the screen based on the target coordinates.

2. The screen handwriting positioning method as described in claim 1, characterized in that, The step of acquiring the handwritten image captured by the camera of the handwriting device in the handwriting direction includes: The handwriting device captures an event image in the handwriting direction using its camera, and sets the event image as the handwriting image, wherein the target screen flashes and displays at least one of the preset encoded images.

3. The screen handwriting positioning method as described in claim 1, characterized in that, The preset encoded image includes a positioning point and an encoded point located within the area of ​​the positioning point. The encoded point has encoded information, which is generated based on the display position information of the corresponding preset encoded image on the target screen. The step of identifying the target encoded image corresponding to the preset encoded image in the handwritten image, and determining the target coordinates of the center of the handwritten image on the target screen based on the target encoded image, includes: The handwritten image is detected to determine at least one target localization point of the target encoded image; For each target encoded image, based on the target localization point of the target encoded image, the target encoding point of the target encoded image in the handwritten image is determined; The target coordinates are determined based on the target encoding information at the target encoding point corresponding to at least one target encoded image and the target positioning point.

4. The screen handwriting positioning method as described in claim 3, characterized in that, The positioning point has a preset coded pattern; The step of detecting the handwritten image and determining the target positioning point in the handwritten image corresponding to the positioning point includes: Based on the preset encoding pattern, the correlation score between the pixels of the handwritten image and the positioning point is calculated; Based on the correlation scores of each pixel in the handwritten image, at least one target location point of the target encoded image is determined.

5. The screen handwriting positioning method as described in claim 4, characterized in that, The step of calculating the correlation score between the pixels of the handwritten image and the preset encoded image based on the preset encoding pattern includes: Based on the preset encoding pattern, a sliding window operation is performed on the handwritten image to calculate the Hamming distance between the sub-window image corresponding to each pixel and the preset encoding pattern. The correlation score of the corresponding pixel is determined based on the Hamming distance.

6. The screen handwriting positioning method as described in claim 4, characterized in that, The preset encoded image includes a first positioning point and a preset number of second positioning points. The first positioning point is located at a preset position, which is determined based on the second positioning points and a preset geometric relationship. Determining the target location point based on the correlation score of each pixel in the handwritten image includes: Based on the correlation scores of each pixel in the handwritten image, a first set of pixels is selected; For each target pixel in the first pixel set, a preset number of first pixels are selected from the first preset range corresponding to the target pixel, wherein the first pixels belong to the first pixel set. Based on the first pixel and the preset geometric relationship, the target position is determined, and the relative distance between the target position and the target pixel is calculated. If the relative distance is less than a preset distance, then the target pixel and the first pixel are set as the target positioning point of a target encoded image.

7. The screen handwriting positioning method as described in claim 6, characterized in that, The preset encoded image includes five positioning points, one of which is located at a preset position that is the midpoint of the diagonal determined based on the other four positioning points.

8. The screen handwriting positioning method as described in claim 6, characterized in that, The step of selecting the first set of pixels based on the correlation scores of each pixel in the handwritten image includes: For each pixel in the handwritten image, if the correlation score corresponding to the pixel is greater than a preset score, then the pixel is recorded in the second pixel set. If the second pixel in the second pixel set has the highest correlation score within its corresponding second preset range, then the second pixel is recorded in the first pixel set to obtain the first pixel set.

9. The screen handwriting positioning method as described in claim 3, characterized in that, The step of determining the target coordinates based on the target encoding information at the target encoding point corresponding to at least one preset encoded image and the target positioning point includes: Based on the camera's intrinsic parameters, calculate the first homography matrix from the region where the target localization point of at least one target-coded image is located to the reference plane. Obtain the coordinates of the reference coding points recorded on the reference plane; The target coding point is determined based on the first homography matrix and the coordinates of the reference coding point.

10. The screen handwriting positioning method as described in claim 3, characterized in that, The step of determining the target coordinates based on the target encoding information at the target encoding point corresponding to at least one preset encoded image and the target positioning point includes: For each target encoded image, target display position information is obtained by processing the target encoding information corresponding to the target encoded image. Based on the target display position information and the target positioning point of the target encoded image, a second homography matrix from the region where the target positioning point is located to the target screen is calculated. The target coordinates are determined based on the second homography matrix corresponding to at least one target encoded image.

11. The screen handwriting positioning method as described in claim 3, characterized in that, The encoding information includes a check code. After determining the target encoding point of the handwritten image in the target encoded image based on the target positioning point of the target encoded image for each target encoded image, the method further includes: Perform non-empty code verification on the target encoding information and obtain the target check code in the target encoding information that has passed the non-empty code verification; Based on the target check code, the validity of the target encoded information for non-empty code verification is checked; Based on the target encoding information that has passed the validity check, the target coordinates are determined by using the target encoding information at the target encoding point corresponding to at least one target encoded image and the target positioning point.

12. The screen handwriting positioning method as described in claim 1, characterized in that, The step of identifying the target encoded image corresponding to the preset encoded image in the handwritten image, and determining the target coordinates of the center of the handwritten image on the target screen based on the target encoded image, includes: Query the display position information of the preset encoded image corresponding to the target encoded image on the target screen; The target coordinates are determined based on the image position information of the target encoded image in the handwritten image and the display position information.

13. The screen handwriting positioning method as described in claim 1, characterized in that, After identifying the target encoded image corresponding to the preset encoded image in the handwritten image, and determining the target coordinates of the center of the handwritten image on the target screen based on the target encoded image, the method further includes: Based on the target coordinates, update the display range of the preset encoded image on the target screen; The target screen is controlled to adjust the display of the preset encoded image according to the changed display range, and then the process returns to the step of acquiring the handwritten image in the handwriting direction captured by the camera of the handwriting device.

14. The screen handwriting positioning method according to any one of claims 1-13, characterized in that, The step of performing screen handwriting positioning on the handwriting device based on the target coordinates includes: The target coordinates are set as the handwriting coordinates of the handwriting device on the target screen.

15. The screen handwriting positioning method according to any one of claims 1-13, characterized in that, The step of performing screen handwriting positioning on the handwriting device based on the target coordinates includes: Obtain the handwriting posture information of the handwriting device; Based on the handwriting posture information and the target coordinates, the handwriting coordinates of the handwriting device on the target screen are determined.

16. A screen handwriting positioning device, characterized in that, The screen handwriting positioning device includes: The acquisition module is used to acquire a handwritten image captured by the camera of the handwriting device in the handwriting direction, wherein the target screen in the handwriting direction displays at least one preset coded image; The recognition module is used to recognize the target encoded image corresponding to the preset encoded image in the handwritten image, and to determine the target coordinates of the center of the handwritten image on the target screen based on the target encoded image; The positioning module is used to perform screen handwriting positioning of the handwriting device based on the target coordinates.

17. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of any of the screen handwriting positioning methods of claims 1-15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the screen handwriting positioning methods of claims 1-15.

Images