Pattern display method and device, electronic equipment and storage medium

Abstract

This application discloses a pattern display method, apparatus, electronic device, and storage medium, belonging to the field of artificial intelligence technology. The method may include receiving a first input, wherein the first input is an input of a user drawing a trajectory, when displaying a first pattern; and displaying at least one drawing intention pattern in response to the first input; wherein the at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and the first pattern; the reference information includes at least one of the following: user chat history, note information; the incremental trajectory includes the trajectory drawn by the user.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a pattern display method, device, electronic device, and storage medium. Background Technology

[0002] With the widespread adoption of touchscreen devices, handwriting input and drawing input have become natural and intuitive human-computer interaction methods. Handwriting input refers to receiving the user's handwritten content via a touchscreen device and converting it into standard text. Drawing input refers to receiving user-drawn patterns or added annotations via a touchscreen device in image editing or chat, and then enhancing them to achieve creative expression.

[0003] In related technologies, text, patterns, or annotations need to be recognized before they can be converted or beautified. However, current recognition methods only perform recognition after the user has completed the drawing and confirmed the input, resulting in the recognition process waiting for the user to finish drawing, poor real-time performance, and reduced efficiency in recognizing user input. Summary of the Invention

[0004] The purpose of this application is to provide a pattern display method, apparatus, electronic device, storage medium, chip, and computer program product that can recognize user input content without waiting for the user to complete the drawing, thereby improving the efficiency of recognizing user input content.

[0005] In a first aspect, embodiments of this application provide a pattern display method, including: When the first pattern is displayed, the first input is received, which is the user's input for drawing the trajectory; In response to the first input, at least one drawing intent pattern is displayed; Wherein, at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and a first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory including the trajectory drawn by the user.

[0006] In this way, by acquiring the corresponding incremental trajectory when the user makes the first input, and combining the incremental trajectory with the first pattern to form the second pattern for recognition, the recognition process does not rely on the complete drawing, significantly shortening the recognition response time and greatly improving the real-time performance of recognizing user input. Moreover, feature extraction and calculation are performed only on the incremental trajectory, eliminating the need to repeatedly calculate the relevant features of the first pattern, effectively reducing computational resource consumption and lowering the device's operating load. Furthermore, after responding to the first input, at least one drawing intent pattern is displayed in real time as a candidate, allowing the user to immediately confirm the desired pattern or continue adjusting the drawing trajectory, avoiding waiting for the complete input before making modifications, further improving the efficiency of user input operations. At the same time, semantic features are constructed by combining user chat history, notes, and other reference information and fused with the graphic features of the second pattern. Based on multimodal features, the user's drawing intent is accurately predicted, making the candidate drawing intent patterns more in line with the user's actual expectations, effectively reducing misidentification and secondary modification operations, and significantly improving the accuracy of recognizing user input.

[0007] In some embodiments of this application, prior to the step of "displaying at least one drawing intention pattern" mentioned above, the pattern display method may further include: Based on the incremental trajectory, determine the geometric incremental feature information corresponding to the incremental trajectory; Based on the geometric increment feature information and the first pattern, determine the graphic feature information of the second pattern; Based on the graphic feature information and reference information of the second pattern, the intended pattern for drawing is determined; The graphic feature information includes geometric feature information and structural feature information. The structural feature information includes the positional information of the shape corresponding to the geometric feature information.

[0008] In this way, the geometric incremental features of the incremental trajectory can be accurately extracted, and combined with the first pattern to integrate complete graphic features including geometry and position structure. With reference information, the drawing intent can be determined from multiple dimensions, which not only reduces redundant calculations and improves recognition efficiency, but also makes the matching of intent patterns more accurate and meets the actual drawing needs of users.

[0009] In some embodiments of this application, the step of "determining the intended drawing pattern based on the graphic feature information and reference information of the second pattern" may specifically include: Based on the graphic feature information of the second pattern, reference patterns that match the graphic feature information of the second pattern are selected from the database. Based on the semantic context information of the reference information, determine the intended drawing pattern from the reference pattern; The semantic context includes intent keywords and intent enhancement feature information; intent keywords are the keywords in the reference information that indicate the user's intent to draw a pattern, and intent enhancement feature information is the information that describes the pattern features indicated by the intent keywords.

[0010] In this way, reference patterns can be accurately screened from the database based on graphic features to narrow down the matching range. Then, the semantic-level secondary screening is performed by combining the intent keywords of the reference information and the intent enhancement information of the pattern features to determine the drawing intent pattern. The dual screening can not only improve the pattern matching efficiency, but also make the results more in line with the user's actual drawing intent, which can greatly improve the recognition accuracy.

[0011] In some embodiments of this application, the database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database. Based on this, the step of "selecting reference patterns that match the graphic feature information of the second pattern from the database according to the graphic feature information of the second pattern" mentioned above may specifically include: Based on the graphic feature information of the second pattern, determine the main graphic and auxiliary graphic in the second pattern; From the reference image database, select the first category of reference images that correspond to the main image; Based on the auxiliary graphics, reference patterns that match the first type of reference graphics are selected from the reference pattern database corresponding to the first type of reference graphics.

[0012] In this way, by splitting the main graphic and auxiliary graphic, and combining the hierarchical reference graphic library and reference pattern library for step-by-step filtering, the main graphic is first determined to narrow the search range, and then the auxiliary graphic is used to accurately match the pattern, which greatly improves the database retrieval efficiency. At the same time, the pattern matching is made to better fit the graphic feature details of the second pattern, thereby improving the matching accuracy.

[0013] In some embodiments of this application, the number of reference patterns is at least two. Based on this, the step of "determining the intended drawing pattern from the reference patterns based on the semantic context information of the reference information" mentioned above may specifically include: Based on the semantic context information of the reference information, determine the user drawing intent score for each of at least two reference patterns; the user drawing intent score is used to characterize the degree of similarity between the reference pattern and the semantic context information of the reference information. At least one reference pattern whose user drawing intent score is greater than or equal to a first threshold is identified as at least one drawing intent pattern.

[0014] In this way, by calculating the intent score for each reference pattern through semantic context, the similarity between the pattern and the user's drawing intent is quantified, and then the matching drawing intent pattern is filtered out by the score threshold. This not only realizes the quantitative judgment of intent matching and improves the objectivity and accuracy of the filtering, but also efficiently filters low-matching patterns, ensuring that the drawing intent pattern is highly consistent with the user's expectations.

[0015] In some embodiments of this application, the reference information includes N text records, where N is an integer greater than or equal to 2. Based on this, before the aforementioned step of "determining the intended drawing pattern from the reference pattern according to the semantic context information of the reference information," the pattern display method may further include: Extract M tokens from N text records, where M is an integer greater than or equal to 2; The lexical weight is determined based on the first frequency, the first quantity, and the second quantity; the first frequency is the frequency of the lexical occurrence in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records that include the lexical. Terminators with a weight greater than or equal to the first threshold are identified as intent keywords for the user's intent pattern in the reference information.

[0016] In this way, by extracting lexical units from multiple text records, combining the frequency of lexical unit occurrence, the total number of texts, and the quantification of texts containing the lexical unit to calculate the lexical unit weight, and then filtering intent keywords according to thresholds, the core semantics in the reference information are accurately extracted, providing accurate semantic basis for subsequent pattern intent matching and improving the matching degree between the drawn intent pattern and the user's semantic needs.

[0017] In some embodiments of this application, prior to the step of "determining the intended drawing pattern from the reference pattern based on the semantic context information of the reference information" mentioned above, the pattern display method may further include: Input the intent keywords into the word vector model and obtain the intent word vectors of the intent keywords output by the word vector model; Input the intent word vector and the lexical weights of the intent keywords into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keywords output by the Chinese pre-trained model; By applying a time decay factor, the semantic enhancement vector is subjected to contextual enhancement processing to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information.

[0018] In this way, the semantic expression of intent keywords is strengthened layer by layer through word vector models and Chinese pre-trained models. Combined with time decay factors, the semantic vectors are enhanced with context, and recent reference information is given higher weight. The intent enhancement feature information that fits the latest intent of the user is accurately extracted, providing more accurate and timely semantic basis for subsequent pattern semantic matching, and greatly improving the matching accuracy between the drawn intent pattern and the user's actual semantic needs.

[0019] In some embodiments of this application, the step of "determining the geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory" mentioned above may specifically include: Adjust the display size of the incremental trajectory to the reference size to obtain the two-dimensional spatial feature information of the incremental trajectory under the reference size; Based on two-dimensional spatial feature information and temporal feature information of incremental trajectory, the spatiotemporal feature map of incremental trajectory is determined; Based on multi-dimensional sensor data corresponding to the incremental trajectory, two-dimensional dynamic feature information is determined; The spatiotemporal feature map and two-dimensional dynamic feature information are input into the convolutional neural network, and the geometric increment feature information corresponding to the increment trajectory is obtained from the output of the convolutional neural network.

[0020] In this way, the incremental trajectory is standardized and its size is adjusted, and the spatiotemporal features are integrated with the dynamic features of multi-dimensional sensors. Then, geometric incremental features are extracted through a convolutional neural network. This not only ensures the standardization and comprehensiveness of feature extraction, but also enables in-depth mining of incremental features with the help of neural networks. This makes the extracted geometric incremental features more accurate and more in line with the actual features of the incremental trajectory, laying a precise feature foundation for subsequent graphic feature integration and pattern matching.

[0021] In some embodiments of this application, prior to the step of "displaying at least one drawing intention pattern" mentioned above, the pattern display method may further include: Obtain the structured trajectory data of the first pattern from the feature cache pool; the structured trajectory data includes historical structured stroke data, which includes the first stroke set. Obtain the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes. From the second set of strokes, select the third set of strokes that does not overlap with the first set of strokes; The incremental trajectory is determined based on the structured stroke data corresponding to each independent stroke in the third stroke set.

[0022] In this way, historical and full-scale structured stroke data are obtained from the feature cache pool and buffer respectively. By filtering the set of differences in strokes, the incremental trajectory is determined, accurately locating the user's newly drawn content and avoiding interference from invalid data. At the same time, historical structured data is reused without repeated processing, reducing the amount of computation and improving the efficiency of incremental trajectory extraction, thus laying a solid, accurate, and efficient data foundation for subsequent feature calculation and intent recognition.

[0023] In some embodiments of this application, prior to the step of "obtaining the full amount of structured stroke data from the first buffer" mentioned above, the pattern display method may further include: The sensor module collects multi-dimensional sensor data of the user's drawn trajectory; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information. Based on multi-dimensional sensor data, at least one independent stroke is determined according to the user's input state when drawing a single stroke; wherein, the input state is used to characterize the state of the user's interactive action when drawing a single stroke. The structured stroke data of each independent stroke is determined based on each independent stroke in at least one independent stroke and the multi-dimensional sensor data corresponding to each independent stroke.

[0024] In this way, by collecting full drawing data such as touch coordinates, pressure sensitivity, speed, and timestamps through multi-dimensional sensors, the user's drawing trajectory features can be fully captured. Based on the drawing input state, individual strokes are broken down, and then structured stroke data is constructed by combining multi-dimensional data. This not only achieves refined and structured storage of drawing trajectory data, but also provides comprehensive and accurate raw data support for subsequent incremental trajectory extraction and feature analysis, ensuring the accuracy of subsequent recognition processes.

[0025] In some embodiments of this application, the step of "determining at least one independent stroke based on multi-dimensional sensor data and the user's input state when drawing a single stroke" may specifically include: Obtain the first multi-dimensional sensor data within the sliding time window from the multi-dimensional sensor data; Based on the user's input state of drawing a single stroke, and using the second multi-dimensional sensor data in the first multi-dimensional sensor data, at least one independent stroke is generated. The second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate independent strokes.

[0026] In this way, incremental sensor data is extracted by sliding time windows, and new independent strokes are generated only based on unused sensor data, accurately capturing the information of strokes newly drawn by the user and avoiding data duplication. At the same time, the real-time and progressive generation of strokes is achieved by relying on sliding windows, improving the timeliness and accuracy of stroke splitting, and providing real-time and accurate stroke data support for subsequent structured data construction and incremental trajectory extraction.

[0027] In some embodiments of this application, the structured stroke data of each independent stroke is stored in a first buffer. Based on this, before the aforementioned step of "obtaining the structured trajectory data of the first pattern from the feature cache pool", the pattern display method may further include: If the amount of structured stroke data stored in the first buffer is greater than or equal to the third threshold, the structured stroke data stored in the first buffer is copied to the second buffer, and the structured stroke data stored in the first buffer is deleted. From the second buffer, obtain the structured stroke data of each individual stroke to determine at least one trajectory drawn by the user; Based on at least one trajectory, determine the geometric feature information corresponding to each trajectory in the at least one trajectory; Based on the geometric feature information corresponding to each trajectory, a first pattern is determined; The structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern are determined as the structured trajectory data of the first pattern, and the structured trajectory data of the first pattern is stored in the feature cache pool.

[0028] In this way, data migration and cleaning are triggered by the buffer threshold, enabling batch processing of the drawing data and avoiding data overload in a single buffer. The trajectory is extracted based on the data in the second buffer, and geometric features are calculated to determine the first pattern. Then, the structured trajectory data and geometric features are stored in the feature cache pool to achieve standardized storage and reuse of historical drawing data. This not only ensures the orderliness of data processing and the efficiency of equipment operation, but also provides accurate and callable historical feature data for subsequent incremental trajectory comparison and graphic feature integration, greatly reducing redundant calculations.

[0029] In some embodiments of this application, the pattern display method may further include: Receive a second input for the intended pattern to be drawn; In response to the second input, based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern, it is determined whether the first graphic drawn by the user is a complete graphic; If the first graphic is incomplete, the third graphic is obtained by completing the first graphic based on the main graphic and auxiliary graphic corresponding to the intended pattern.

[0030] In this way, the system receives and responds to secondary input of the intended drawing pattern, first determines the integrity of the graphic, and then, for incomplete graphics, intelligently completes the graphic based on the main and auxiliary graphics of the intended drawing pattern to obtain a new graphic. This not only achieves accurate determination of graphic integrity, but also can selectively complete graphic details based on matching intention features, so that the completed graphic fits the user's drawing intention, improves graphic expression, enhances the integrity and accuracy of graphic drawing, and reduces the user's manual completion operation cost.

[0031] In some embodiments of this application, the pattern display method may further include: Receive third input for a stop drawing control or for a drawing intention pattern; In response to the third input, an image frame for each stroke in the user-drawn stroke set is generated based on the stroke set of the first graphic drawn by the user. A pattern drawing video is generated based on the image frame of each stroke and the image frame corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first or last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern; Play images to draw videos.

[0032] In this way, in response to the input of stopping drawing or drawing an intention pattern, the system generates user stroke image frames in the drawing order and combines them with the intention pattern image frames to create a drawing video. The first or last frame is the intention pattern frame and the video is played. This not only restores the user's hand-drawing process and intuitively presents the process of generating a beautiful pattern from a doodle, but also enriches the display of the drawing results through video, improves the interactive experience and the visualization effect of the results. At the same time, no additional user operation is required, and the video can be generated and played with one click, reducing the operation threshold.

[0033] In some embodiments of this application, the drawing intent pattern includes a first type of drawing intent pattern and a second type of drawing intent pattern. Based on this, the pattern display method may further include: The drawing intention pattern of the first type of painting style is displayed according to the first display size, and the drawing intention pattern of the second type of painting style is displayed according to the second display size; Among them, the first display size is larger than the second display size; the confidence level of the first type of painting style is greater than that of the second type of painting style.

[0034] In this way, different display sizes are matched to different styles of intent patterns based on the confidence level. High-confidence patterns are displayed in larger sizes, while low-confidence patterns are displayed in smaller sizes, allowing users to quickly focus on the best matching results, improving the visual recognition and selection efficiency of intent patterns, while clearly distinguishing candidate solutions with different confidence levels, taking into account both selection diversity and ease of operation.

[0035] Secondly, embodiments of this application provide a pattern display device, including: The receiving module is used to receive a first input when the first pattern is displayed, wherein the first input is the user's input of drawing the trajectory; The display module is used to display at least one drawing intent pattern in response to the first input; Wherein, at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and a first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory including the trajectory drawn by the user.

[0036] In this way, by acquiring the corresponding incremental trajectory when the user makes the first input, and combining the incremental trajectory with the first pattern to form the second pattern for recognition, the recognition process does not rely on the complete drawing, significantly shortening the recognition response time and greatly improving the real-time performance of recognizing user input. Moreover, feature extraction and calculation are performed only on the incremental trajectory, eliminating the need to repeatedly calculate the relevant features of the first pattern, effectively reducing computational resource consumption and lowering the device's operating load. Furthermore, after responding to the first input, at least one drawing intent pattern is displayed in real time as a candidate, allowing the user to immediately confirm the desired pattern or continue adjusting the drawing trajectory, avoiding waiting for the complete input before making modifications, further improving the efficiency of user input operations. At the same time, semantic features are constructed by combining user chat history, notes, and other reference information and fused with the graphic features of the second pattern. Based on multimodal features, the user's drawing intent is accurately predicted, making the candidate drawing intent patterns more in line with the user's actual expectations, effectively reducing misidentification and secondary modification operations, and significantly improving the accuracy of recognizing user input.

[0037] In some embodiments of this application, the pattern display device may further include a determining module for determining geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory; Based on the geometric increment feature information and the first pattern, determine the graphic feature information of the second pattern; Based on the graphic feature information and reference information of the second pattern, the intended pattern for drawing is determined; The graphic feature information includes geometric feature information and structural feature information. The structural feature information includes the positional information of the shape corresponding to the geometric feature information.

[0038] In this way, the geometric incremental features of the incremental trajectory can be accurately extracted, and combined with the first pattern to integrate complete graphic features including geometry and position structure. With reference information, the drawing intent can be determined from multiple dimensions, which not only reduces redundant calculations and improves recognition efficiency, but also makes the matching of intent patterns more accurate and meets the actual drawing needs of users.

[0039] In some embodiments of this application, the pattern display device may further include a filtering module for filtering reference patterns that match the graphic feature information of the second pattern from a database based on the graphic feature information of the second pattern. The pattern display device may further include a determining module for determining a drawing intention pattern from a reference pattern based on the semantic context information of the reference information; The semantic context includes intent keywords and intent enhancement feature information; intent keywords are the keywords in the reference information that indicate the user's intent to draw a pattern, and intent enhancement feature information is the information that describes the pattern features indicated by the intent keywords.

[0040] In this way, reference patterns can be accurately screened from the database based on graphic features to narrow down the matching range. Then, the semantic-level secondary screening is performed by combining the intent keywords of the reference information and the intent enhancement information of the pattern features to determine the drawing intent pattern. The dual screening can not only improve the pattern matching efficiency, but also make the results more in line with the user's actual drawing intent, which can greatly improve the recognition accuracy.

[0041] In some embodiments of this application, the pattern display device may further include a determining module, used to determine the main graphic and auxiliary graphic in the second pattern based on the graphic feature information of the second pattern, when the database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database. The pattern display device may also include a filtering module for filtering a first type of reference graphic corresponding to the main graphic from a reference graphic database; The filtering module can also be used to filter reference patterns that match the first type of reference pattern from a database of reference patterns corresponding to the first type of reference pattern, based on the auxiliary graphic.

[0042] In this way, by splitting the main graphic and auxiliary graphic, and combining the hierarchical reference graphic library and reference pattern library for step-by-step filtering, the main graphic is first determined to narrow the search range, and then the auxiliary graphic is used to accurately match the pattern, which greatly improves the database retrieval efficiency. At the same time, the pattern matching is made to better fit the graphic feature details of the second pattern, thereby improving the matching accuracy.

[0043] In some embodiments of this application, the pattern display device may further include a determination module, configured to determine a user drawing intent score value for each of the at least two reference patterns based on the semantic context information of the reference information, when the number of reference patterns is at least two; the user drawing intent score value is used to characterize the degree of similarity between the reference pattern and the semantic context information of the reference information. At least one reference pattern whose user drawing intent score is greater than or equal to a first threshold is identified as at least one drawing intent pattern.

[0044] In this way, by calculating the intent score for each reference pattern through semantic context, the similarity between the pattern and the user's drawing intent is quantified, and then the matching drawing intent pattern is filtered out by the score threshold. This not only realizes the quantitative judgment of intent matching and improves the objectivity and accuracy of the filtering, but also efficiently filters low-matching patterns, ensuring that the drawing intent pattern is highly consistent with the user's expectations.

[0045] In some embodiments of this application, the pattern display device may further include an extraction module for extracting M word units from N text records, where M is an integer greater than or equal to 2, when the reference information includes N text records, where N is an integer greater than or equal to 2. The pattern display device may further include a determining module for determining the word weight of a word based on a first frequency, a first quantity, and a second quantity; the first frequency is the frequency of the word appearing in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records including the word. The determination module can also be used to identify words with weights greater than or equal to a first threshold as intent keywords for user-drawn intent patterns in the reference information.

[0046] In this way, by extracting lexical units from multiple text records, combining the frequency of lexical unit occurrence, the total number of texts, and the quantification of texts containing the lexical unit to calculate the lexical unit weight, and then filtering intent keywords according to thresholds, the core semantics in the reference information are accurately extracted, providing accurate semantic basis for subsequent pattern intent matching and improving the matching degree between the drawn intent pattern and the user's semantic needs.

[0047] In some embodiments of this application, the pattern display device may further include an acquisition module for inputting intent keywords into a word vector model and acquiring the intent word vectors of the intent keywords output by the word vector model; The pattern display device may also include a processing module for inputting the lexical weights of the intent word vector and the intent keyword into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keyword output by the Chinese pre-trained model; The processing module can also be used to perform contextual enhancement processing on the semantic enhancement vector through a time decay factor to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information.

[0048] In this way, the semantic expression of intent keywords is strengthened layer by layer through word vector models and Chinese pre-trained models. Combined with time decay factors, the semantic vectors are enhanced with context, and recent reference information is given higher weight. The intent enhancement feature information that fits the latest intent of the user is accurately extracted, providing more accurate and timely semantic basis for subsequent pattern semantic matching, and greatly improving the matching accuracy between the drawn intent pattern and the user's actual semantic needs.

[0049] In some embodiments of this application, the pattern display device may further include an adjustment module for adjusting the display size of the incremental trajectory to a reference size, thereby obtaining two-dimensional spatial feature information of the incremental trajectory at the reference size. The pattern display device may also include a determining module for determining the spatiotemporal feature map of the incremental trajectory based on two-dimensional spatial feature information and temporal feature information of the incremental trajectory; The determination module can also be used to determine two-dimensional dynamic feature information based on multi-dimensional sensor data corresponding to the incremental trajectory; The pattern display device may also include a processing module for inputting spatiotemporal feature maps and two-dimensional dynamic feature information into a convolutional neural network, and obtaining geometric increment feature information output by the convolutional neural network corresponding to the increment trajectory.

[0050] In this way, the incremental trajectory is standardized and its size is adjusted, and the spatiotemporal features are integrated with the dynamic features of multi-dimensional sensors. Then, geometric incremental features are extracted through a convolutional neural network. This not only ensures the standardization and comprehensiveness of feature extraction, but also enables in-depth mining of incremental features with the help of neural networks. This makes the extracted geometric incremental features more accurate and more in line with the actual features of the incremental trajectory, laying a precise feature foundation for subsequent graphic feature integration and pattern matching.

[0051] In some embodiments of this application, the pattern display device may further include an acquisition module for acquiring structured trajectory data of a first pattern from a feature cache pool; the structured trajectory data includes historical structured stroke data, and the historical structured stroke data includes a first stroke set; The acquisition module can also be used to acquire the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes. The pattern display device may also include a filtering module for filtering a third set of strokes from the second set of strokes that does not overlap with the first set of strokes; The pattern display device may also include a determination module for determining an incremental trajectory based on structured stroke data corresponding to each individual stroke in the third stroke set.

[0052] In this way, historical and full-scale structured stroke data are obtained from the feature cache pool and buffer respectively. By filtering the set of differences in strokes, the incremental trajectory is determined, accurately locating the user's newly drawn content and avoiding interference from invalid data. At the same time, historical structured data is reused without repeated processing, reducing the amount of computation and improving the efficiency of incremental trajectory extraction, thus laying a solid, accurate, and efficient data foundation for subsequent feature calculation and intent recognition.

[0053] In some embodiments of this application, the pattern display device may further include a data acquisition module for acquiring multi-dimensional sensor data of the user-drawn trajectory through a sensor module; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information; The pattern display device may further include a determining module for determining at least one independent stroke based on multi-dimensional sensor data and according to the user's input state when drawing a single stroke; wherein the input state is used to characterize the state of the user's interactive action when drawing a single stroke. The determination module can also be used to determine the structured stroke data of each independent stroke based on each independent stroke in at least one independent stroke and the multi-dimensional sensor data corresponding to each independent stroke.

[0054] In this way, by collecting full drawing data such as touch coordinates, pressure sensitivity, speed, and timestamps through multi-dimensional sensors, the user's drawing trajectory features can be fully captured. Based on the drawing input state, individual strokes are broken down, and then structured stroke data is constructed by combining multi-dimensional data. This not only achieves refined and structured storage of drawing trajectory data, but also provides comprehensive and accurate raw data support for subsequent incremental trajectory extraction and feature analysis, ensuring the accuracy of subsequent recognition processes.

[0055] In some embodiments of this application, the pattern display device may further include an acquisition module for acquiring first multi-dimensional sensor data within a sliding time window from multi-dimensional sensor data; The pattern display device may also include a generation module for generating at least one independent stroke based on second multi-dimensional sensor data in the first multi-dimensional sensor data, according to the user's input state of drawing a single stroke. The second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate independent strokes.

[0056] In this way, incremental sensor data is extracted by sliding time windows, and new independent strokes are generated only based on unused sensor data, accurately capturing the information of strokes newly drawn by the user and avoiding data duplication. At the same time, the real-time and progressive generation of strokes is achieved by relying on sliding windows, improving the timeliness and accuracy of stroke splitting, and providing real-time and accurate stroke data support for subsequent structured data construction and incremental trajectory extraction.

[0057] In some embodiments of this application, the pattern display device may further include a storage module, which is used to copy the structured stroke data stored in the first buffer to a second buffer and delete the structured stroke data stored in the first buffer when the structured stroke data of each independent stroke exists in a first buffer and the amount of structured stroke data stored in the first buffer is greater than or equal to a third threshold. The pattern display device may also include a determination module for obtaining structured stroke data of each individual stroke from a second buffer and determining at least one trajectory drawn by the user; The determination module can also be used to determine the geometric feature information corresponding to each trajectory in at least one trajectory, based on at least one trajectory. The determination module can also be used to determine the first pattern based on the geometric feature information corresponding to each trajectory; The storage module can also be used to determine the structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern as the structured trajectory data of the first pattern, and store the structured trajectory data of the first pattern in the feature cache pool.

[0058] In this way, data migration and cleaning are triggered by the buffer threshold, enabling batch processing of the drawing data and avoiding data overload in a single buffer. The trajectory is extracted based on the data in the second buffer, and geometric features are calculated to determine the first pattern. Then, the structured trajectory data and geometric features are stored in the feature cache pool to achieve standardized storage and reuse of historical drawing data. This not only ensures the orderliness of data processing and the efficiency of equipment operation, but also provides accurate and callable historical feature data for subsequent incremental trajectory comparison and graphic feature integration, greatly reducing redundant calculations.

[0059] In some embodiments of this application, the receiving module may also be used to receive a second input to the intended drawing pattern; The pattern display device may further include a determining module for determining, in response to a second input, whether the first graphic drawn by the user is a complete graphic based on the main graphic and auxiliary graphic corresponding to the intended drawing pattern; The pattern display device may also include a completion module, which, when the first graphic is incomplete, completes the first graphic based on the main graphic and auxiliary graphic corresponding to the intended pattern to obtain a third graphic.

[0060] In this way, the system receives and responds to secondary input of the intended drawing pattern, first determines the integrity of the graphic, and then, for incomplete graphics, intelligently completes the graphic based on the main and auxiliary graphics of the intended drawing pattern to obtain a new graphic. This not only achieves accurate determination of graphic integrity, but also can selectively complete graphic details based on matching intention features, so that the completed graphic fits the user's drawing intention, improves graphic expression, enhances the integrity and accuracy of graphic drawing, and reduces the user's manual completion operation cost.

[0061] In some embodiments of this application, the receiving module can also be used to receive a third input to the stop drawing control or to the drawing intent pattern; The pattern display device may also include a generation module for generating an image frame of each stroke in the user-drawn stroke set in response to a third input, based on the stroke set of the user-drawn first graphic. The generation module can also be used to generate a pattern drawing video based on the image frame of each stroke and the image frame corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first or last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern; The pattern display device may also include a playback module for playing video of the image drawing.

[0062] In this way, in response to the input of stopping drawing or drawing an intention pattern, the system generates user stroke image frames in the drawing order and combines them with the intention pattern image frames to create a drawing video. The first or last frame is the intention pattern frame and the video is played. This not only restores the user's hand-drawing process and intuitively presents the process of generating a beautiful pattern from a doodle, but also enriches the display of the drawing results through video, improves the interactive experience and the visualization effect of the results. At the same time, no additional user operation is required, and the video can be generated and played with one click, reducing the operation threshold.

[0063] In some embodiments of this application, the display module can also be used to display the drawing intention pattern of the first type of painting style according to a first display size, and to display the drawing intention pattern of the second type of painting style according to a second display size, when the drawing intention pattern includes a drawing intention pattern of the first type of painting style and a drawing intention pattern of the second type of painting style. Among them, the first display size is larger than the second display size; the confidence level of the first type of painting style is greater than that of the second type of painting style.

[0064] In this way, different display sizes are matched to different styles of intent patterns based on the confidence level. High-confidence patterns are displayed in larger sizes, while low-confidence patterns are displayed in smaller sizes, allowing users to quickly focus on the best matching results, improving the visual recognition and selection efficiency of intent patterns, while clearly distinguishing candidate solutions with different confidence levels, taking into account both selection diversity and ease of operation.

[0065] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the pattern display method as shown in the first aspect.

[0066] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the pattern display method as shown in the first aspect are implemented.

[0067] Fifthly, embodiments of this application provide a chip, which includes a processor and a display interface. The display interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the pattern display method as shown in the first aspect.

[0068] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the pattern display method as described in the first aspect.

[0069] In this embodiment, when displaying a first pattern, a first input is received, which is the user's input of a drawn trajectory. In response to the first input, at least one drawing intent pattern is displayed. This at least one drawing intent pattern is determined based on a second pattern and reference information. The second pattern is composed of an incremental trajectory corresponding to the first input and the first pattern. The reference information includes at least one of the following: user chat history, note information, and the incremental trajectory includes the trajectory drawn by the user. Thus, by obtaining the corresponding incremental trajectory when the user makes the first input, and combining the incremental trajectory with the first pattern to form the second pattern for recognition, the recognition process does not rely on complete drawing, significantly shortening the recognition response time and greatly improving the real-time performance of recognizing user input. Furthermore, feature extraction and calculation are performed only on the incremental trajectory portion, eliminating the need for repeated calculations of features related to the first pattern, effectively reducing computational resource consumption and lowering the device's operating load. Moreover, displaying at least one drawing intent pattern as a candidate object in real time after responding to the first input allows the user to immediately confirm the desired pattern or continue adjusting the drawing trajectory, avoiding waiting for complete input before making modifications, further improving the efficiency of user input operations. At the same time, semantic features are constructed by combining user chat history, notes and other reference information and fused with the graphic features of the second pattern. Based on multimodal features, the user's drawing intention is accurately predicted, making the candidate drawing intention pattern more in line with the user's actual expectations, effectively reducing misidentification and secondary modification operations, and significantly improving the accuracy of recognizing user input content.

[0070] It is understood that the pattern display device provided in the second aspect, the electronic device provided in the third aspect, the readable storage medium provided in the fourth aspect, the chip provided in the fifth aspect, and the computer program product provided in the sixth aspect are all used to execute the method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0071] Figure 1 Flowcharts of pattern display methods provided for some embodiments of this application; Figure 2 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 3 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 4 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 5 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 6 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 7 A schematic diagram of the interface for a pattern display method provided in some embodiments of this application; Figure 8 A schematic diagram of the structure of a pattern display device provided for some embodiments of this application; Figure 9 A schematic diagram of the structure of an electronic device is provided for some embodiments of this application; Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided for some embodiments of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0073] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. Embodiments of this application provide a pattern display method, apparatus, electronic device, and storage medium. The following, in conjunction with the appendix... Figures 1 to 10 The pattern display method provided in this application will be described in detail through specific embodiments and application scenarios.

[0074] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0075] The terminology used in the embodiments of this invention will be explained below.

[0076] Handwriting input method refers to an input method that receives user-written content through the touch screen of an electronic device and converts it into standard text. Specifically, the handwriting input method receives text written by the user.

[0077] A graffiti control is a UI element used in image editing or chat applications to receive user-drawn patterns or annotations, which can be enhanced to achieve creative expression.

[0078] The first trajectory drawn by the user refers to the content drawn by the user in the first time period, which is an incremental trajectory based on the content drawn in the second time period.

[0079] User-drawn content refers to text, patterns, or annotations drawn by users and received via the touchscreen of an electronic device.

[0080] A Convolutional Neural Network (CNN) is a deep learning model used for image recognition and processing, which automatically extracts image features through convolutional layers. In this embodiment, it is used to extract the geometric increment feature information of the image covering the content area corresponding to the first trajectory.

[0081] The Term Frequency-Inverse Document Frequency (TF-IDF) algorithm is a weighted technique used in information retrieval and text mining that calculates the importance of terms by using term frequency inverse document frequency.

[0082] The term frequency-inverse document frequency weight refers to the numerical value calculated by the term frequency-inverse document frequency algorithm, which represents the importance of a term in text information.

[0083] A word embedding model is a technique that maps words or phrases in natural language to dense vectors, so that semantically similar words are close in distance in the vector space. It can capture the contextual relationships and semantic information between words. In this embodiment, the word embedding model may include the Word2Vec model.

[0084] Chinese pre-trained models refer to models that are pre-trained on large-scale Chinese corpora and can be transferred to various Chinese Natural Language Processing (NLP) tasks to obtain semantic vectors of words.

[0085] In this application embodiment, a stroke refers to the smallest drawing unit formed by a single continuous touch operation of the user. It is the basic element that constitutes the trajectory. It is represented by the drawing line corresponding to a complete interactive action of the user from pressing the pen on the screen, moving the drawing, to lifting the pen. It has independent structured data such as coordinates, pressure sensitivity, speed, timestamp, etc., and cannot be further divided.

[0086] The trajectory in this embodiment refers to the dynamic drawing traces generated during the user's on-site drawing process, which is an ordered collection of strokes. It is intuitively represented by all the continuous or discontinuous lines drawn by the user in real time in the drawing area. It is a direct dynamic expression of the drawing behavior, existing only during the drawing process, without undergoing feature extraction, recognition, and processing in the background.

[0087] In this application embodiment, the shape refers to the abstract geometric feature representation of the trajectory and strokes that is not publicly displayed by the backend technology layer. It is an intermediate product of backend recognition. The abstract geometric shape formed by the backend extracting the geometric features of the trajectory, such as contour, size, structure, and position, has no visual display form and is only used as the technical basis for backend matching, completion, and pattern generation.

[0088] In this application's embodiments, the graphic refers to a complete geometric visual form formed by intelligent processing of shapes in the background, such as completion, correction, and standardization, like a circle, square, or triangle. It is a visual and standardized representation of the shape. It can be understood as a standardized visual carrier obtained after the background refines the abstract shape, possessing complete and standardized geometric features, and serving as an intermediate visual layer connecting abstract shapes and figurative patterns.

[0089] The pattern in this application refers to the final visual result displayed after the user completes the drawing. It is a concrete visual product formed by combining graphics with stylized rendering and semantic matching. Based on the graphic framework, it integrates painting styles such as cartoons, oil paintings, and two-dimensional elements, as well as semantic intents such as suns and smiley faces combined with chat records. It is the final product presented to the outside world after backend processing, and it is also the final object that the user can perceive, select, and use.

[0090] It should be noted that in this embodiment, strokes are the basic units, forming trajectories step by step. A stroke is the smallest indivisible unit of graffiti drawing; multiple ordered strokes combine according to the user's drawing order to form the trajectory drawn by the user on-site. The trajectory is a collection of strokes, its characteristics determined by the structured data of all the strokes that make it up. The trajectory is the original basis of the shape, and the shape is the abstract result of the trajectory. The backend uses the trajectory drawn by the user on-site as the data source, extracting its geometric features such as outline, structure, and position, and abstracting them into a shape. The shape is not displayed externally, but only serves as an intermediate basis for backend technical processing; it is the result of transforming the trajectory from a dynamic trace visible to the user to geometric features recognizable by the backend. The shape is then refined to form a graphic, which is the standardized visualization of the shape. Based on the extracted shape, the backend performs intelligent completion, correction, and standardization processing, such as completing incomplete circles and correcting skewed lines, to form a graphic. The graphic is the visual representation of the shape, possessing a complete and standardized geometric form, solving the problems of incompleteness and non-standardization of the original trajectory and shape. The graphic is the basic framework of the pattern, and the pattern is the concrete finished product of the graphic. Using graphics as the core geometric framework, combined with semantic matching results, painting style rendering, and other elements, a pattern is ultimately generated. The pattern is a product of the graphic's further processing and is the final visual result that the user sees and can use after drawing. It realizes the transformation from standardized geometric forms to concrete visual products that fit the user's intentions. In other words, the strokes form the trajectory drawn by the user on-site in sequence. The backend abstracts the undisplayed shape from the trajectory, refines the shape to form a standardized graphic, and then processes the graphic as a basis to obtain the final displayed pattern.

[0091] Painting style refers to an image that possesses a specific artistic style. This specific artistic style can include at least one of the following: oil painting, watercolor, cartoon, or anime / manga.

[0092] It should be noted that the pattern display method provided in this application can be executed by electronic devices such as mobile phones, tablets, laptops, PDAs, and wearable devices, or by servers. Some embodiments of this application use electronic devices as the executing entity to illustrate the pattern display method provided in this application. By recognizing users' messy scribbles in real time and automatically converting them into refined, complete objects such as text, patterns, or adding annotations, this solves the problems of high barriers to entry and low efficiency in traditional input methods for graphic input, allowing ordinary users to quickly input personalized visual content without professional drawing skills.

[0093] The pattern display method provided in this application embodiment can be applied to any of the following scenarios: chat hand-drawn emoticon scenarios, handwritten meeting minutes scenarios, online meeting real-time annotation scenarios, engineering students' drawing scenarios, and document annotation scenarios, as detailed below. Scenario 1: Chat hand-drawn emoticon scenarios. In this scenario, a user wants to send a custom cartoon sun emoticon in a chat. Since there is no matching emoticon in the existing database, the user opens the graffiti control through their input method to enable the graffiti input function and creates their own custom sun emoticon by hand. During the drawing process, the electronic device recognizes and displays the results in real time. After completion, the user's hand-drawn sun pattern is displayed, and different drawing styles such as oil painting, anime, and cartoon are also provided for the user to choose from. If no style is actively selected, the most suitable style will be matched based on the chat context.

[0094] Scenario 2: Handwritten Meeting Minutes. In this scenario, when users handwrite meeting minutes, they may want to use a personalized text style that matches the meeting theme or their personal preferences. If their existing input method cannot meet this need, they can open a handwriting input method and start writing. During the writing process, the electronic device analyzes and displays the preliminary recognition results in real time. After writing is complete, it displays both the converted original handwritten text and provides different font styles for the user to choose from, such as formal business, creative handwriting, and simple modern. If no font style is actively selected, the most suitable default style will be matched based on the meeting context, such as formal reports or brainstorming sessions.

[0095] Scenario 3: Real-time annotation in online meetings. In online meetings, participants need to quickly express their opinions but lack suitable annotation tools. They can then access the drawing control on the meeting interface to enable hand-drawn input mode, providing real-time feedback through drawing. The recognition results are simultaneously displayed on the shared screen during the drawing process. When drawing basic symbols such as "√", "×", and "?", they will be interpreted as corresponding marks of agreement, disagreement, or questions requiring clarification, and presented visually accordingly. When discussing different meeting topics such as budget allocation and project progress, doodles will be prioritized for recognition as graphics relevant to the topic. If no graphic type is specified, the appropriate graphic semantics will be matched based on keywords from the meeting minutes.

[0096] Scenario 4: Drawing scenario for engineering students. In engineering drawing studies, students need to quickly draw professional sketches and add dimensions, but lack standard drawing tools. They then enable the drawing software's doodle control to start a hand-drawn input mode. During the hand-drawn engineering sketch process, standardized processing results are simultaneously displayed on the canvas. When drawing basic engineering symbols such as roughness and welding, they are automatically converted to corresponding national standards or standard annotation styles. Drawing the outline of a 3D shape generates matching three-view drawings, and hand-drawn arrows and numerical dimension annotations are standardized to conform to drafting specifications. When drawing different types of drawings such as mechanical parts drawings and architectural floor plans, doodles are prioritized for recognition as professional symbols matching the drawing type. If the annotation type is not clearly defined, the appropriate engineering symbols are matched based on layer attributes and graphic features.

[0097] Scenario 5: Document Annotation. When users quickly annotate a document, they can use the drawing input function to complete the annotation operation. When drawing basic symbols such as "√", "×", and "?", they will be converted into corresponding aesthetic annotation marks in real time. Drawing circles and underlines will be recognized as highlight or key marks that are accurately aligned with the document. Handwritten text and graphic annotations will be automatically aesthetically enhanced into clear standard fonts and neat graphics, and will still retain editability after being embedded in the document.

[0098] Based on this, the following is combined with Figure 1 A pattern display method provided in the embodiments of this application will be described in detail.

[0099] Figure 1 A flowchart of a pattern display method provided for some embodiments of this application.

[0100] like Figure 1 As shown, the pattern display method provided in this application embodiment can be applied to electronic devices. Based on this, the pattern display method may include steps 110 to 130, as detailed below.

[0101] Step 110: When the first pattern is displayed, receive the first input, which is the user's input for drawing the trajectory.

[0102] The first pattern can be an incomplete form during the pattern drawing process. For example, if a user wants to draw the sun, the first image can include the basic circular pattern of the sun, but not the radial lines outside the basic circle. Of course, in other embodiments, the first pattern can also be a complete form, such as including both the basic circular pattern of the sun and the radial lines outside the basic circle.

[0103] The first input includes, but is not limited to: touch input from the user via a finger or stylus on an electronic device, voice commands input by the user, specific gestures input by the user, or other feasible inputs. The specific input can be determined according to actual usage needs, and this embodiment of the invention does not impose limitations. The specific gestures in this application embodiment can be any one of the following: single-click gesture, swipe gesture, drag gesture, pressure recognition gesture, long-press gesture, area change gesture, double-press gesture, and double-tap gesture. Click input or selection input in this application embodiment can be single-click input, double-click input, or any number of clicks, and can also be long-press input or short-press input. The first input in this application includes, but is not limited to, user input via handwriting input method and user input via drawing controls in the input method.

[0104] In some embodiments, taking the above-described scenario one as an example, such as Figure 2 As shown, when the input method interface 201 displays the first pattern, such as the basic circular pattern 202, in order to draw a unique cartoon sun emoticon, the user continues to draw the trajectory of radial lines next to the circular pattern by hand. The electronic device receives the user's first input of this hand-drawn radial lines.

[0105] In other embodiments, taking the above-mentioned scenario two as an example, when the handwriting input interface displays a first pattern such as the meeting title "Project Promotion Meeting", in order to supplement the meeting minutes, the user hand-draws the text trajectory of "Meeting Time" below the title, and the electronic device receives the user's first input of the handwritten meeting content.

[0106] It should be noted that the prerequisite for executing step 110 in this embodiment is that the electronic device is running an input method application and is in an interactive state capable of receiving user input, such as a chat input scenario. The user performs a trigger operation on the graffiti control in the input method interface to enable the graffiti input function. In response to the user's operation on the graffiti control, the graffiti interface is displayed, which includes at least a pen setting area, a graffiti drawing area, and a style selection area. The graffiti interface is in an editable state, allowing the user to input the drawing trajectory through touch, handwriting, or other methods. In this way, the electronic device is in a state of detecting user drawing input and is ready to collect the drawing trajectory data generated by the user in the graffiti drawing area.

[0107] It should be noted that users can start editing their doodles on the doodle interface provided by the doodle input function. A diagram of the doodle interface is shown below. Figure 3As shown. The top area 301 of the graffiti interface allows users to select different colors and brush thicknesses, as well as use an eraser to modify the graffiti. It also allows for one-click clearing and undoing of the previous action. The middle area 302 of the graffiti interface is the drawing area. The bottom area 303 of the graffiti interface allows users to select drawing styles, such as oil painting, anime, or cartoon.

[0108] Step 120: In response to the first input, display at least one drawing intent pattern; wherein the at least one drawing intent pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and the first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory including a trajectory drawn by the user.

[0109] In this embodiment of the application, the note information is text containing personal notes made by the user, such as in a notebook or memo.

[0110] In some embodiments, taking the above-described scenario one as an example, such as Figure 4 As shown, in response to the user's first input of drawing radial lines next to the basic circle, the electronic device immediately displays at least one drawing intention pattern such as a cartoon sun, a watercolor sun, or a two-dimensional sun; wherein, these drawing intention patterns are determined based on a second pattern and the user's chat history, the second pattern is composed of the incremental trajectory of the radial lines corresponding to the first input and the basic circle pattern 202, the reference information being the chat history where the user previously mentioned "wanting to draw a cute sun to send a message", the incremental trajectory of the circle outline may include the radial lines drawn by the user this time.

[0111] It should be noted that the second pattern can be an incomplete pattern consisting of a basic circle and three radiating lines, such as... Figure 5 As shown, it can also be a complete second pattern consisting of a basic circle and 6 radiating lines. If the user has already drawn the basic circle and 3 radiating lines, it can also be... Figure 4 The display area 303 displays three drawing intent patterns, which can be intelligently completed based on the basic circle and three radial lines to obtain a new graphic. This not only achieves accurate judgment of graphic integrity, but also completes graphic details based on matching intent features, making the completed graphic fit the user's drawing intent, improving graphic expression, enhancing the integrity and accuracy of graphic drawing, and reducing the user's manual completion operation cost.

[0112] In other embodiments, taking the above-described scenario two as an example, in response to the user's first input of drawing the words "Meeting Time" on the blank writing area of ​​the handwriting input method, the electronic device immediately displays at least one drawing intention pattern, such as "Project Promotion Meeting" in neat printed font, formal business font, or simplified handwriting font, as well as "Meeting Time: January 1, 2026". These drawing intention patterns are determined based on a second pattern and the user's meeting notes information. The second pattern consists of the incremental trajectory of "Meeting Time" corresponding to the first input and "Project Promotion Meeting", and the reference information is the user's notes that "Project Promotion Meeting needs to be formally recorded".

[0113] In some embodiments of this application, the pattern display method may further include steps 1201 to 1203 before step 120 described above.

[0114] Step 1201: Determine the geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory.

[0115] In this embodiment, geometric incremental feature information refers to a set of pure geometric features obtained after standardization and feature extraction of the incremental drawing trajectory generated by user input. This set is used to characterize the inherent geometric attributes of the incremental trajectory's lines, including basic geometric parameters such as line length, line direction, curvature, line density, arrangement angle, contour shape, and thickness. It is a quantitative representation of the shape of the newly added hand-drawn trajectory itself. The geometric incremental feature information in this embodiment includes at least one of the following: circumscribed rectangle incremental features, shape descriptor incremental features, and contour incremental features. Specifically, the circumscribed rectangle incremental features refer to the incremental boundary constraint features obtained by fitting a circumscribed rectangle to the overall contour of the incremental trajectory, including rectangle length and width, rectangle deflection angle, coordinate boundaries, and area proportions. The shape descriptor incremental features refer to standardized shape representation parameters formed by encoding the curvature, direction density, thickness variation, and contour topology of the incremental trajectory lines. The contour incremental features refer to the incremental geometric information specific to the line contour, such as the trend of the incremental trajectory's edge lines, the distribution of contour inflection points, the degree of contour closure, and changes in edge curvature. In this step, if the first pattern is a basic circle, the user draws radial lines next to the circle to form an incremental trajectory. Based on this, the electronic device can extract information such as the direction, length, arrangement angle, density and shape of the radial lines to determine the corresponding geometric incremental feature information.

[0116] Step 1202: Determine the graphic feature information of the second pattern based on the geometric increment feature information and the first pattern.

[0117] In this step, the geometric incremental feature information of the radial lines is combined with the geometric features of the basic circle as the first pattern, as well as the relative arrangement position and overall combination structure relationship between the circle and the radial lines. The geometric features and the structural features corresponding to the positions are integrated to jointly determine the complete graphic feature information of the second pattern composed of the basic circle and the radial lines.

[0118] In this embodiment, graphic feature information refers to comprehensive overall feature data generated by fusing the inherent geometric features of the first pattern and the geometric incremental feature information of the incremental trajectory, while encompassing both geometric and structural feature information. The graphic feature information includes the overall outline, size, proportions, and other global geometric shapes of the first pattern combined with the incremental trajectory. It also includes the relative coordinates, arrangement positions, hierarchical relationships, and combination connections between various local shapes. The graphic feature information is used to characterize the overall shape and spatial relationship of the second pattern, providing a complete feature basis for matching the user's intended drawing pattern with reference information.

[0119] Step 1203: Based on the graphic feature information and reference information of the second pattern, determine the intended drawing pattern; wherein, the graphic feature information includes geometric feature information and structural feature information, and the structural feature information includes the position information of the shape corresponding to the geometric feature information.

[0120] In this step, the graphic feature information of the second pattern's fusion position structure and outline shape is used as a basis for comprehensive analysis, and the semantic information of the user's chat history is combined to accurately match and finally determine the cartoon sun, oil painting sun, and other drawing intention patterns that meet the user's needs.

[0121] In this way, the geometric incremental features of the incremental trajectory can be accurately extracted, and combined with the first pattern to integrate complete graphic features including geometry and position structure. With reference information, the drawing intent can be determined from multiple dimensions, which not only reduces redundant calculations and improves recognition efficiency, but also makes the matching of intent patterns more accurate and meets the actual drawing needs of users.

[0122] The steps 1201 to 1203 mentioned above will be described in detail below.

[0123] Regarding step 1201, in some embodiments of this application, the geometric increment feature information in the embodiments of this application can be three-dimensional information. The following describes in detail how to construct three-dimensional geometric increment feature information. Based on this, the above step 1201 may specifically include steps 12011 to 12014.

[0124] Step 12011: Adjust the display size of the incremental trajectory to the reference size to obtain the two-dimensional spatial feature information of the incremental trajectory under the reference size.

[0125] In this step, a two-dimensional X and Y coordinate system is established to normalize the size of the incremental trajectory, scaling the content coverage area of ​​the incremental trajectory to a standard reference size, such as 128×128. Based on this, the touch coordinate points constituting all the strokes of the incremental trajectory are first collected, and the overall bounding box of the incremental trajectory is calculated; the extreme values ​​of the coordinates are extracted to determine the original actual size range of the incremental trajectory; while keeping the aspect ratio unchanged, the scaling factor is calculated, and a smaller scaling ratio is selected to prevent line distortion; the original coordinates of each stroke of the incremental trajectory are sequentially translated and scaled to uniformly map to the standard two-dimensional space corresponding to the reference size; finally, the calibrated and normalized complete set of stroke coordinates is output, which is the two-dimensional spatial feature information of the incremental trajectory under the reference size.

[0126] Step 12012: Based on the two-dimensional spatial feature information and the temporal feature information of the incremental trajectory, determine the spatiotemporal feature map of the incremental trajectory.

[0127] In this step, a three-dimensional tensor with dimensions of [height, width, time step] is first constructed. All stroke data under each time step and the time step corresponding to the incremental trajectory are traversed. A corresponding two-dimensional spatial feature map is generated for each time step. State marking is performed at the effective pixel positions along the path of the incremental stroke, and the coordinate boundaries are checked to ensure that the pixel coordinates are within the legal range. Then, Gaussian filtering is used to simulate the thickness and edge transition effect of real hand-drawn strokes. The two-dimensional spatial feature maps processed at each time step are stacked and stored in the time dimension level corresponding to the three-dimensional tensor. Thus, a three-dimensional tensor structure with two-dimensional space and one-dimensional time is constructed. Here, X and Y constitute the basic dimensions of the planar space, and time t is introduced as an independent temporal dimension. Finally, the spatiotemporal feature map of the incremental trajectory, which integrates the spatial position and temporal sequence relationship, is obtained. The spatiotemporal feature map of the incremental trajectory can be represented as F_spatial_temporal(x,y,t)={stroke state of coordinates (x,y) at time t}.

[0128] Step 12013: Based on the multi-dimensional sensor data corresponding to the incremental trajectory, determine the two-dimensional dynamic feature information. The multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information. The two-dimensional dynamic feature information can be represented by F_dynamic.

[0129] In this step, the touch movement speed information is calculated based on the screen touch position pressure sensitivity information from multi-dimensional sensor data, combined with the touch position coordinate information and the corresponding timestamp sequence information. This together constructs a dual-feature channel of pressure sensitivity and speed, which can be represented as F_dynamic=[P_normalized,V_normalized]. Specifically, this can be determined through the following steps: First, traverse the original multi-dimensional sensor data sequence corresponding to each stroke of the incremental trajectory, simultaneously acquiring screen touch position coordinate information, screen touch position pressure sensitivity information, and matching timestamp sequence information; normalize the original pressure sensitivity values, for example, using the maximum range value such as 1024 as a benchmark to complete amplitude normalization; using the coordinate difference between adjacent touch position coordinates and the time difference between the corresponding timestamp sequence information, solve the instantaneous speed of the touch movement point by point, and set the initial speed of the first acquisition point to 0; then, constrain the range of the speed value using the hyperbolic tangent function, mapping it to […]. The standard interval is defined as 1,1]. Finally, the normalized pressure sensitivity features and the standardized velocity features are fused and combined, and output in vector form to obtain the two-dimensional dynamic feature information corresponding to the incremental trajectory.

[0130] It should be noted that this example uses the construction of pressure sensitivity and speed dual feature channels. In this embodiment, the trajectory curvature and movement speed dual feature channels can also be constructed based on the timestamp sequence information corresponding to the screen touch position coordinates, which can be represented as F_dynamic=[Curvature_normalized, Velocity_normalized]. Specifically, first, the screen touch position coordinates and timestamp sequence information corresponding to each stroke of the incremental trajectory are traversed. Based on the curve trend of the continuous coordinate points, the local curvature of the trajectory is calculated point by point to obtain the original curvature value. The curvature value range is then uniformly normalized. Then, the instantaneous speed of the touch movement is solved by combining the difference between adjacent touch coordinates and the difference between timestamps. The speed of the first sampling point is set to 0. The speed is mapped to […] using the hyperbolic tangent function. [1,1] standard interval; combine and splice the normalized trajectory curvature features with the standardized movement speed features, and output them in vector form to finally obtain the two-dimensional dynamic feature information of curvature-related pen movement speed under the incremental trajectory.

[0131] Step 12014: Input the spatiotemporal feature map and two-dimensional dynamic feature information into the convolutional neural network, and obtain the geometric increment feature information corresponding to the increment trajectory output by the convolutional neural network.

[0132] In this step, a convolutional neural network can be used to fuse spatiotemporal features and two-dimensional dynamic features to extract multi-scale deep features. These multi-scale deep features can be represented as: F_geometry=CNN([F_spatial_temporal,F_dynamic]). Based on this, ResNet or EfficientNet can be used as the basic architecture, ultimately outputting a 512-dimensional feature vector representing geometric increment feature information. Specifically, the convolutional neural network can output geometric increment feature information corresponding to the increment trajectory through the following steps: First, a pre-trained EfficientNet is selected as the backbone feature extraction network; the first convolutional layer of the network is adapted to be compatible with multi-channel combined input; the output structure of the network ends is adjusted to limit the network output to a 512-dimensional feature vector corresponding to the increment trajectory; a multi-head attention module is integrated into the network to enhance the feature expression ability of key contours and trend details. Secondly, average pooling is used to compress the temporal dimension of the spatiotemporal feature map. The compressed spatiotemporal feature information is then concatenated and fused with the two-dimensional dynamic feature information along the channel dimension. Following network layout specifications, the feature tensor arrangement is converted from "batch, height, width, channel" to the standard format of "batch, channel, height, width." If the channel dimension after fusion is insufficient, additional edge and texture features corresponding to the incremental trajectory are calculated to enrich the representation. Multi-level high-level semantic feature extraction is then performed via a backbone convolutional network. The edge feature calculation method involves using the Sobel operator to solve for the gradient information in the horizontal X-direction and vertical Y-direction respectively. The mean of the input features along the channel dimension is calculated before convolution, and the edge intensity features of the incremental trajectory contour are obtained based on the gradient magnitude. The texture feature calculation method uses the standard deviation statistics of local regions as the basis for texture description, statistically analyzing the standard deviation distribution along the channel dimension to characterize the local texture fluctuation characteristics caused by the density of lines and stroke variations in the incremental trajectory. Based on this, the geometric incremental feature information precisely corresponding to the incremental trajectory is obtained through the integrated deduction output of the aforementioned convolutional neural network.

[0133] In this way, the incremental trajectory is standardized and its size is adjusted, and the spatiotemporal features are integrated with the dynamic features of multi-dimensional sensors. Then, geometric incremental features are extracted through a convolutional neural network. This not only ensures the standardization and comprehensiveness of feature extraction, but also enables in-depth mining of incremental features with the help of neural networks. This makes the extracted geometric incremental features more accurate and more in line with the actual features of the incremental trajectory, laying a precise feature foundation for subsequent graphic feature integration and pattern matching.

[0134] In step 1202, in this embodiment of the application, the incremental trajectory can be determined based on structured stroke data corresponding to at least one independent stroke, and the geometric incremental feature information of the incremental trajectory can be represented as FeatureCache={Stroke_1:F ;Stroke_2:F ;...;Stroke_n:F }, where FeatureCache refers to an object or dictionary used to cache the geometric incremental feature information of the incremental trajectory. Stroke_i represents the i-th independent stroke, i∈[1,n], for example, Stroke_1 represents the first independent stroke, F The structured stroke data for the i-th independent stroke can be represented by a vector. This vector can include the structured stroke data of the independent stroke, which may include multi-dimensional sensor data of the independent stroke, such as screen touch position coordinates, screen touch pressure sensitivity, touch movement speed, and timestamp sequence information corresponding to the screen touch position coordinates. Based on this, newly added strokes since the last determination of geometric increment feature information can be represented as ΔF={F_{n+1},F_{n+2},...}. Therefore, feature extraction is only performed on newly added strokes, without recalculating all stroke features, which reduces the dependence on subsequent complete drawing and reduces computational resource consumption.

[0135] The geometric increment feature information ΔF can be fused with the first pattern using the following formula (1) to obtain the graphic feature information F_total_new of the second pattern, as shown below.

[0136] The Aggregate function can be a preset weighted average, max pooling, or attention-weighted. FeatureCache can be the first pattern. In this embodiment, the geometric increment feature information can include the bounding rectangle increment feature, the shape descriptor increment feature, and the contour increment feature. Based on this, the graphic feature information of the second pattern can include the updated bounding rectangle increment feature, which can be represented as BoundingBox_new=Union(BoundingBox_old,BoundingBox_delta), where BoundingBox_old is the bounding rectangle feature in the first pattern, and BoundingBox_delta is the increment data of the bounding rectangle in the geometric increment feature information.

[0137] Based on this, we will take the scenario of a user drawing a sun as an example. The first pattern is the basic circular pattern displayed after the user finishes drawing it. The incremental trajectory consists of three radial lines drawn by hand next to the basic circle to supplement the sun's rays. All three strokes are straight lines formed by continuous touch. The geometric incremental feature information of the incremental trajectory is the circumscribed rectangle incremental feature, shape descriptor incremental feature, and contour incremental feature extracted from the three radial lines. These are all features that are not displayed but extracted by the backend. Based on this, combined with the geometric incremental feature information and the basic circular pattern, the graphic feature information of the second pattern, i.e., the complete sun graphic, is determined, clarifying the complete geometric shape of the sun graphic and providing a standardized geometric framework for subsequent stylized rendering.

[0138] Based on the incremental trajectory drawn by the user, namely the three radial lines, the specific geometric incremental feature information that can be extracted is as follows, all of which are undisplayed intermediate products from the background. The incremental feature of the bounding rectangle of the three radial lines can include: the first radial line: bounding rectangle coordinates are x1=150, y1=80; x2=180, y2=50, width 30 pixels, height 30 pixels, and deflection angle 30°; the second radial line: bounding rectangle coordinates are x1=180, y1=100; x2=210, y2=100, width 30 pixels, height 5 pixels, and deflection angle 0°; the third radial line: bounding rectangle coordinates are x1=150, y1=120; x2=180, y2=150, width 30 pixels, height 30 pixels, and deflection angle -30°; the incremental feature of the bounding rectangle of the three combined lines: the bounding rectangle coordinates after merging are x1=150, y1=50; x2=210, y2=150, width 60 pixels, height 100 pixels, and deflection angle 0°. The incremental features of the shape descriptor include: line type (straight lines with zero curvature); size parameters (single stroke length 30 pixels, line thickness 2 pixels); and arrangement angles (the three lines form angles of 30°, 0°, and -30° with the center of the base circle, respectively, and are evenly distributed on the outer side of the base circle). The incremental features of the contour include three radial lines with straight contours, no curves, no holes, smooth contour edges, no obvious burrs, a spacing of 30 pixels between adjacent lines, and a minimum distance of 10 pixels from the contour of the base circle.

[0139] Based on the displayed basic circle, and combined with the specific values ​​of the above-mentioned geometric incremental feature information, standardized completion and correction are performed to determine the complete sun graphic feature information. Specifically, it can include the first pattern and the circular outline with a radius of 50 pixels, the center coordinates of the circle being (180, 100), the line thickness of 2 pixels, and the color being yellow. The geometric features based on the incremental trajectory include completion into 6 radial lines. Based on the original arrangement angle of 3 lines, they are evenly completed into 6 lines, with each line having an angle of 60°. The length of each line is uniformly corrected to 30 pixels, and the thickness is consistent with the basic circle. All 6 lines start from the center (180, 100) and extend outwards, with the endpoint coordinates being (210, 100), (195, 70), (165, 70), (150, 100), (165, 130), and (195, 130), respectively. The distance between the lines and the basic circular outline is uniformly 10 pixels. All lines are straight lines with smooth outlines, no burrs, and the angles between adjacent lines are consistent, forming a uniform radial pattern. The second pattern is a sun graphic consisting of a circle, for example, with a radius of 50 pixels, and six evenly radiating straight lines, for example, each line is 30 pixels. The center coordinates of the circle are (180, 100). The coordinates of the overall circumscribed rectangle are x1=120, y1=70; x2=240, y2=130, with a width of 120 pixels and a height of 60 pixels. The geometric shape is complete and standardized.

[0140] In this way, through the extraction and standardization of specific numerical geometric incremental feature information, the three radial trajectories drawn by the user are transformed into standardized and complete sun graphic feature information. This clarifies the specific dimensions, positions, and arrangement angles of the graphic, providing a precise geometric framework for subsequently generating sun patterns in different styles, such as cartoon and oil painting styles. Simultaneously, by completing the number of lines and standardizing their size and spacing, the standardization of graphic features is ensured, avoiding the influence of irregularities in the user's drawn trajectories, such as uneven stroke thickness and angular deviations, on the final pattern display effect. This approach both aligns with the user's drawing intentions and guarantees the consistency and accuracy of graphic features.

[0141] It should be noted that in the above embodiment, the trajectory consists of three radial lines drawn by the user on-site, representing real-time drawing traces without background processing. Each stroke is a single touch from pressing to releasing, representing the smallest unit of the trajectory. The shape is an abstract geometric feature of the three trajectories extracted by the background (such as line angles, lengths, and spacing, i.e., an abstract representation of the geometric incremental feature information mentioned earlier), not displayed, but only used as a basis for determining graphic features. The graphic is the standardized geometric shape of the sun, i.e., a circle and six radial lines, representing a visually standardized representation of the shape. The pattern is the final sun pattern generated subsequently using stylized rendering, such as a cartoon style; it is the concrete result of the graphic and the content ultimately displayed and used by the user.

[0142] Regarding step 1203, in this embodiment of the application, step 1203 may specifically include steps 12031 and 12032, as shown below.

[0143] Step 12031: Based on the graphic feature information of the second pattern, select reference patterns from the database that match the graphic feature information of the second pattern.

[0144] In this step, the basic shape of the first pattern can be used for quick classification to determine that the basic shape of the second pattern is a circle, corresponding to relevant patterns under the circle category in the database, excluding patterns with non-circular bases such as square or polygonal patterns. Next, the focus is on matching the characteristics of the circular base and radial lines to filter out reference patterns that include a circular body and radial structure, excluding patterns without radial features such as simple circles or patterns without radial lines. In this way, three categories of reference patterns that meet the requirements can be filtered out, such as sun and star decorative icons and radial warning signs.

[0145] Step 12032: Determine the drawing intention pattern from the reference pattern based on the semantic context information of the reference information.

[0146] In this step, the semantic context of the user's chat history can be used to accurately match the user's drawing intent from the reference patterns selected in step 12031, thus determining the final drawing intent pattern. Specifically, the user's chat history includes explicit semantic information, namely, the intent keyword is "sun," the intent enhancement feature is "radiating rays, warm color tone," and the semantic context can be "the user needs a sun emoji with radiating rays and a warm color tone." Based on this, the three selected reference patterns—the sun, the starlight decorative icon, and the radial warning sign—are matched with the semantic context information. The cartoon sun includes a circular base, for example, a radius of 50 pixels, a center coordinate of 180,100, and six radial lines evenly distributed around the circular body, for example, line length of 30 pixels and thickness of 2 pixels. The color tone is warm yellow, and it is a commonly used style for emojis. It perfectly matches the intent enhancement features of the keyword "sun," "radiating rays," and "warm color tone," meeting the semantic requirements of "warm color tone and emoji scene" in the user's chat history. The starlight decorative icon and the radial warning sign are excluded. Based on this, the cartoon sun can be identified as the intended design from among the three reference designs: the sun, the starlight decorative icon, and the radial warning sign.

[0147] In this way, reference patterns can be accurately screened from the database based on graphic features to narrow down the matching range. Then, the semantic-level secondary screening is performed by combining the intent keywords of the reference information and the intent enhancement information of the pattern features to determine the drawing intent pattern. The dual screening can not only improve the pattern matching efficiency, but also make the results more in line with the user's actual drawing intent, which can greatly improve the recognition accuracy.

[0148] Furthermore, in step 12031 above, the intended pattern can be filtered and drawn through a two-level architecture. Based on this, the database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database. Based on this, step 12031 may specifically include steps 120311 to 12033.

[0149] Step 120311: Based on the graphic feature information of the second pattern, determine the main graphic and auxiliary graphic in the second pattern.

[0150] In this step, the graphic features of the second pattern are represented by a circular outline and radial lines. The entire circular area can be used as the main graphic, which reflects the user's drawing intent and serves as the basic framework for subsequent stylized rendering. Correspondingly, the detailed elements surrounding the main graphic, namely the radial lines, can be used as auxiliary graphics to complete the overall graphic structure. Together with the main graphic, they form a complete graphic feature to meet the needs of subsequent stylized rendering.

[0151] Step 120312: From the reference graphic database, select the first type of reference graphic that corresponds to the main graphic.

[0152] In this step, the first-level architecture performs rapid classification of basic shapes. Its purpose is to quickly categorize incremental trajectories into predefined basic shape categories, narrowing the retrieval space and reducing the computational overhead of subsequent graphic matching. The reference graphic database used in this architecture is a basic shape library. This library predefines a complete set of multi-class reference graphics, namely BaseShapes={circle, ellipse, rectangle, triangle, pentagram, heart, straight line, curve, spiral, polygon, ...}, covering various commonly used basic geometric shapes and meeting the classification needs of different graffiti scenarios.

[0153] This process can employ lightweight classifiers such as SVM or shallow neural networks for fast classification. The classification expression is: BaseShape_class = FastClassifier(F_geometry_basic). Here, F_geometry_basic represents the geometric features of the main graphic, such as bounding rectangle features, shape descriptor features, and contour features. The output is the first-class reference graphic corresponding to the main graphic. The first-class reference graphic can include at least one category. If only one category of reference graphic exists, such as a circle, it is directly used as the first-class reference graphic for subsequent reference pattern matching. If multiple categories of reference graphics exist, the first three categories and their confidence probabilities are output. For example, BaseShape_candidates = [(circle, 0.8), (ellipse, 0.15), (heart, 0.05)]. Based on this, the category with the highest confidence probability, such as a circle, can be selected as the first-class reference graphic, balancing classification accuracy and processing efficiency.

[0154] It should be noted that during the process of users drawing doodles, they often only input part of the trajectory, and the pattern is not yet completed. In order to achieve incremental and real-time basic shape prediction, in the rapid classification stage of basic shapes, it is necessary to predict the probability of its belonging to various basic shape categories based on local trajectory features. Based on this, the probability distribution of which reference graphic the user's independent stroke belongs to can be determined according to the following formulas (2) and (3): Where P(Class|Partial_Strokes) represents the posterior probability that the pattern formed under the condition of the strokes (Partial_Strokes) currently drawn by the user belongs to a certain reference graphic (Class), that is, the probability that the model judges what kind of reference graphic it is most likely when only looking at the lines you have drawn halfway. F_partial is the structured stroke data extracted from the strokes. Softmax(Classifier(F_partial)) refers to converting the classifier output into a probability distribution through the Softmax function. Classifier(F_partial): The linear transformation output of the classifier to F_partial. Softmax: Converts the linear output into a probability distribution in the range of [0,1], ensuring that the sum of the probabilities of all classes is 1. Based on this, the class with high confidence can be selected as the first type of reference graphic by using the confidence score. The confidence score can be calculated by the following formula (3).

[0155] Where P(Class_i) is the probability of the i-th class, and Confidence is the confidence level of the i-th class. A confidence threshold, such as 0.7, can be set, and based on this, classes with a confidence level higher than 0.7 are used as the first type of reference graphics. In this embodiment, the confidence threshold can be dynamically adjusted, specifically through the following steps: In the low confidence stage, recognition is triggered once for every additional stroke; in the high confidence stage, recognition is triggered once for every three additional strokes. This avoids over-computation and improves efficiency. Moreover, in some embodiments, the reference pattern and reference information matching the class with the highest current confidence can be used to determine the drawing intention pattern, so that the user can click on it, thereby reducing user input. If the user does not click, the displayed drawing intention pattern can be updated in real time as the user continues to draw. Step 120313: Based on the auxiliary graphics, in the reference pattern database corresponding to the first type of reference graphics, filter the reference patterns that match the first type of reference graphics.

[0156] For example, the second-level architecture is intra-class fine-grained matching, the purpose of which is to further match specific icons or emoticons corresponding to auxiliary graphics within the determined basic shape categories. Each basic shape category corresponds to an independent sub-database, namely the reference pattern database, used to store various patterns or images with the first category of reference graphics, for example, IconLibrary[circle] = {smiley face, sun, basketball, earth, clock, ...}. Specifically, deep neural networks can be used to extract the geometric feature information of the auxiliary graphics, and the similarity between the auxiliary graphics and the reference patterns in the reference pattern database can be calculated using the following formula (4): Where F_icon_i represents the geometric feature information of the i-th reference pattern in the reference pattern database, and F_multimodal represents the geometric feature information of the auxiliary graphic. Based on this, an approximate nearest neighbor search algorithm such as FAISS is used to accelerate similarity calculation to obtain the similarity scores of the top-10 intra-class patterns and intra-class patterns in the sub-database. Based on the above similarity calculation results, an approximate nearest neighbor search algorithm is used to accelerate retrieval and sorting, so that the reference pattern with the highest similarity to the auxiliary graphic in the reference pattern database is determined as the reference pattern matching the first type of reference graphic.

[0157] In this way, by splitting the main graphic and auxiliary graphic, and combining the hierarchical reference graphic library and reference pattern library for step-by-step filtering, the main graphic is first determined to narrow the search range, and then the auxiliary graphic is used to accurately match the pattern, which greatly improves the database retrieval efficiency. At the same time, the pattern matching is made to better fit the graphic feature details of the second pattern, thereby improving the matching accuracy.

[0158] Furthermore, in conjunction with steps 120311 to 12033 above, the number of reference patterns is at least two. In step 12032 above, the intention pattern can be filtered and drawn through the third-level architecture. Based on this, step 12032 above can specifically include steps 120321 and 120322.

[0159] Step 120321: Based on the semantic context information of the reference information, determine the user drawing intent score value for each of the at least two reference patterns; the user drawing intent score value is used to characterize the similarity between the reference pattern and the semantic context information of the reference information. The semantic context information of the reference information can be determined once at the start of user drawing and remains unchanged during the user drawing process. It does not need to be repeatedly calculated and can be directly read from the cache of the electronic device, effectively reducing the amount of computational resources required.

[0160] In this step, the semantic relevance score (Semantic_score(Icon_i)) of the i-th reference pattern can be calculated using the following formula (5): Wherein, V_context represents the semantic context of the text information, and V_icon_semantic_i represents the semantic context of the i-th in-class pattern. V_icon_semantic_i is pre-annotated or generated through image description. Next, the content shape similarity between the i-th reference pattern and the user-drawn pattern and the semantic relevance of the reference pattern can be fused using the following formula (6) to obtain the user drawing intent score of the reference pattern, as shown below: Where α and β are weighting coefficients, set to α=0.6 and β=0.4, Similarity_score(Icon_i) is the content shape similarity between the i-th reference pattern and the user-drawn pattern, and Final_score(Icon_i) is the user's drawing intent score for the i-th reference pattern.

[0161] If the semantic context information is represented as "weather, sunny day, sun", then semantic similarity scores are calculated for the sun, smiley face, clock, basketball, etc. in the reference pattern to obtain the corresponding user drawing intent score values, such as 0.92 for the sun, 0.65 for the smiley face, 0.31 for the clock, and 0.18 for the basketball. It should be noted that in the embodiments of this application, the user drawing intent score value can be calculated by the cosine similarity algorithm or the Bidirectional Encoder Representations from Transformers (BERT) semantic similarity algorithm.

[0162] Step 120322: Determine at least one reference pattern whose user drawing intent score is greater than or equal to a first threshold as at least one drawing intent pattern.

[0163] In this step, a first threshold is preset, for example, to 0.6, and the user drawing intent score obtained in the preceding steps is compared with this first threshold. Based on the user drawing intent score calculated above, at least two reference patterns can be reordered using the following formula (7): Based on the aforementioned ranking, reference patterns with user drawing intent scores greater than or equal to a first threshold are determined as the final drawing intent patterns. Continuing with the example above, if the first threshold is 0.6, then the sun (0.92) and smiley face (0.65) both meet the criteria and can be selected as drawing intent patterns; while the clock and basketball are discarded because their scores are below the first threshold. This method allows for precise selection of the target pattern that best matches the user's actual drawing intent from multiple candidate reference patterns, improving the accuracy of subsequent rendering or recommendation results.

[0164] In this way, by calculating the intent score for each reference pattern through semantic context, the similarity between the pattern and the user's drawing intent is quantified, and then the matching drawing intent pattern is filtered out by the score threshold. This not only realizes the quantitative judgment of intent matching and improves the objectivity and accuracy of the filtering, but also efficiently filters low-matching patterns, ensuring that the drawing intent pattern is highly consistent with the user's expectations.

[0165] This application also provides a process for determining the semantic context information of reference information. Semantic context information refers to information used to characterize the scenario in which the user's current drawing behavior occurs, the purpose of drawing, historical interaction habits, associated text prompts, application environment tags, and related semantic relationships. It reflects the user's potential intent and semantic preferences at the current drawing stage. This information can semantically constrain and assist in determining the type of pattern the user expects to draw, providing a semantic basis for selecting reference patterns that match the user's true drawing intent. Based on this, the steps for determining intent keywords and intent enhancement feature information are described in detail below.

[0166] In some embodiments of this application, the reference information may include N text records, where N is an integer greater than or equal to 2. Based on this, before step 12032 above, the pattern display method may also include steps 2101 to 2103.

[0167] Step 2101: Extract M tokens from N text records, where M is an integer greater than or equal to 2.

[0168] In this step, the eight most recent text records are extracted from the reference information, such as chat logs in a chat context, including messages between the user and the other party, such as... Figure 6 As shown, in a hypothetical chat log scenario, a user draws a cartoon sun in N text messages. The tokens extracted from each message can be as follows: Figure 7 As shown. It should be noted that if there are fewer than 10 chat records, all usable records will be extracted, and the chat records will be preprocessed, such as removing emoticons and special characters, while retaining valid text.

[0169] Step 2102: Determine the word weight of the word based on the first frequency, the first quantity, and the second quantity; the first frequency is the frequency of the word appearing in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records that include the word.

[0170] In this step, as described above... Figure 6 He Ru Figure 7 Taking the example of the word "sun" as an example, the first quantity H1 is 8 messages. The number of messages containing the word "sun" is 3, that is, the second quantity H2 is 3, namely messages 3, 4, and 5. The first frequency of the word "sun" in message 3 is P: 1 / 2 = 0.5, the first frequency of the word "sun" in message 4 is P: 1 / 4 = 0.25, and the first frequency of the word "sun" in message 5 is P: 1 / 3 = 0.33.

[0171] Based on this, the lexical weight TF-IDF(w) of lexical w can be extracted using the TF-IDF algorithm according to the following formula (8): For example, the term weight of TF_IDF_Sun_Message3 is 0.5×log(8 / 3)=0.5×0.426=0.213; the term weight of TF_IDF_Sun_Message4 is 0.25×log(8 / 3)=0.25×0.426=0.107; and the term weight of TF_IDF_Sun_Message5 is 0.33×log(8 / 3)=0.33×0.426=0.71.

[0172] Step 2103: Identify the words with a weight greater than or equal to the first threshold as the intent keywords for the user to draw intent patterns in the reference information.

[0173] For example, the word weight results can be as follows: "sun": 0.213; "drawing": 0.189; "cartoon": 0.156; "smiley face": 0.72; "radiance": 0.72; "cute": 0.128; "glasses": 0.107; "mouth": 0.107; "sunshine": 0.095; "weather": 0.089. Based on this, words with weights greater than or equal to the first threshold, such as 0.72, are selected as intent keywords to construct the intent keywords: Keywords={kw ,kw ,...,kw For example, if the word weight is greater than or equal to the first threshold, such as 0.72, the intent keyword is Keywords={“sun”, “painting”, “cartoon”, “smiley face”, “radiance”}.

[0174] In this way, by extracting lexical units from multiple text records, combining the frequency of lexical unit occurrence, the total number of texts, and the quantification of texts containing the lexical unit to calculate the lexical unit weight, and then filtering intent keywords according to thresholds, the core semantics in the reference information are accurately extracted, providing accurate semantic basis for subsequent pattern intent matching and improving the matching degree between the drawn intent pattern and the user's semantic needs.

[0175] In some embodiments of this application, prior to step 12032 above, the pattern display method may further include steps 2201 to 2203 to determine the intended enhancement feature information.

[0176] Step 2201: Input the intent keywords into the word vector model and obtain the intent word vectors of the intent keywords output by the word vector model.

[0177] For example, a pre-trained word vector model, namely the Word2Vec model, can be used to convert intent keywords into intent word vectors. The output intent word vectors of intent keywords are shown below: Word2Vec_vectors={"Sun":[0.8,0.6,-0.2,0.9,0.3],"Drawing":[0.1,0.4,0.7,-0.1,0.5],"Cartoon":[0.5,0.2,0.8,0.4,-0.3],"Smiling face":[0.7,0.9,0.1,0.6,0.2],"Radiance":[0.9,0.3,-0.1,0.8,0.4]}.

[0178] Step 2202: Input the intent word vector and the lexical weights of the intent keywords into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keywords output by the Chinese pre-trained model.

[0179] For example, the word weights of the intent keywords can be as follows: weights={"sun":0.213,"painting":0.189,"cartoon":0.156,"smileyface":0.72,"radiance":0.72}. Based on this, a Chinese pre-trained model with a dimension of 200 is adopted. This Chinese pre-trained model can perform a weighted average of the word vectors of multiple keywords using the following formula (9): Where weight_i represents the TF-IDF weight of the intent keyword, calculated numeratorically for each of the 200 dimensions. Example of weighted average calculation: numerator=[0.213×0.8+0.189×0.1+0.156×0.5+0.72×0.7+0.72×0.9 #1st dimension] 0.213×0.6+0.189×0.4+0.156×0.2+0.72×0.9+0.72×0.3#Second dimension 0.213×(-0.2)+0.189×0.7+0.156×0.8+0.72×0.1+0.72×(-0.1)#3rd dimension 0.213×0.9+0.189×(-0.1)+0.156×0.4+0.72×0.6+0.72×0.8#4th ​​dimension 0.213×0.3+0.189×0.5+0.156×(-0.3)+0.72×0.2+0.72×0.4#5th dimension……] The calculation results are illustrated using a 5-dimensional example: numerator=[0.465,0.434,0.218,0.398,0.222]. The denominator, i.e., the sum of weights, is denominator=0.213+0.189+0.156+0.72+0.72=0.842.

[0180] The semantic vector of the final intent keyword can be represented as: V_context=[0.465 / 0.842,0.434 / 0.842,0.218 / 0.842,0.398 / 0.842,0.222 / 0.842]; V_context=[0.552,0.515,0.259,0.473,0.264].

[0181] Step 2203: Perform context enhancement processing on the semantic enhancement vector using a time decay factor to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information (intent keyword). The intent enhancement feature information can be represented by V_context_final.

[0182] For example, in this embodiment of the application, a time decay factor can be introduced to give higher weight to recent messages. Based on this, the semantic vector of the intent keyword can be enhanced with the following formula (10): Where Δt is the time difference between the text record of the intent keyword and the semantic vector of the intent keyword output by the Chinese pre-trained model, and λ is the attenuation coefficient. The intent enhancement feature information can be represented as V_context_final=Σ(weight_time_i×V_context_i), where the intent enhancement feature information can be represented by a vector.

[0183] The parameters in the above formula (10) will be explained below with a specific example.

[0184] Time decay factor calculation: Assuming the current time is t=0, the time difference between messages is: Message time difference (minutes): Message 1: Δt =15 minutes Message 2: Δt =12 minutes Message 3: Δt =8 minutes Message 4: Δt =6 minutes Message 5: Δt =4 minutes Message 6: Δt =3 minutes Message 7: Δt =2 minutes Message 8: Δt =1 minute The attenuation coefficient λ = 0.1, which is an empirical value.

[0185] The calculation process for time weights is as follows: weight_time =exp(-0.1×15)=exp(-1.5)=0.223 weight_time =exp(-0.1×12)=exp(-1.2)=0.301 weight_time =exp(-0.1×8)=exp(-0.8)=0.449 weight_time =exp(-0.1×6)=exp(-0.6)=0.549 weight_time =exp(-0.1×4)=exp(-0.4)=0.670 weight_time =exp(-0.1×3)=exp(-0.3)=0.741 weight_time =exp(-0.1×2)=exp(-0.2)=0.819 weight_time =exp(-0.1×1)=exp(-0.1)=0.905 Assume the semantic vector V_context_i for each message has already been calculated. Here, we take the first dimension as an example: V_context_final[0]=( 0.223×V_context [0]+0.301×V_context [0]+0.449×V_context [0]+ 0.549×V_context [0]+0.670×V_context [0]+0.741×V_context [0]+ 0.819×V_context [0]+0.905×V_context [0] ) / (0.223+0.301+0.449+0.549+0.670+0.741+0.819+0.905) Based on this, the total weight is 4.657, that is, V_context_final=[0.628,0.543,0.312,0.567,0.389,...,0.445]#200-dimensional vector.

[0186] In this way, the semantic expression of intent keywords is strengthened layer by layer through word vector models and Chinese pre-trained models. Combined with time decay factors, the semantic vectors are enhanced with context, and recent reference information is given higher weight. The intent enhancement feature information that fits the latest intent of the user is accurately extracted, providing more accurate and timely semantic basis for subsequent pattern semantic matching, and greatly improving the matching accuracy between the drawn intent pattern and the user's actual semantic needs.

[0187] In some embodiments of this application, prior to step 120, the pattern display method may further include steps 2301 to 2304 to determine the incremental trajectory.

[0188] Step 2301: Obtain the structured trajectory data of the first pattern from the feature cache pool; the structured trajectory data includes historical structured stroke data, and the historical structured stroke data includes the first stroke set.

[0189] Step 2302: Obtain the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes.

[0190] Step 2303: From the second set of strokes, select the third set of strokes that does not overlap with the first set of strokes.

[0191] Step 2304: Determine the incremental trajectory based on the structured stroke data corresponding to each independent stroke in the third stroke set.

[0192] In this embodiment, a dual-buffering mechanism can be employed. One buffer, the first buffer, is used for data acquisition, and the other buffer, the second buffer, is used for feature extraction to avoid data contention. Specifically, the first buffer can be set to a maximum storage capacity of 1024 sampling points, maintain a count of the currently stored data, and mark whether the data is currently being written. The second buffer can also be set to the same maximum storage capacity of 1024 sampling points, maintain the amount of data in the current buffer, and mark whether feature extraction processing is currently underway. A switching mechanism is provided between the two buffers: when the amount of data in the first buffer is greater than or equal to a first threshold, the structured data stored in the first buffer can be added to the second buffer.

[0193] Taking the rendering of the sun as an example, when a user draws a circular subject, at the start of the drawing process, the first buffer begins receiving data from the first contact point, including coordinates, pressure, velocity, and timestamp. In subsequent moments, the first buffer continuously receives data from each contact point during the drawing process, recording changes in coordinates, pressure, velocity, and corresponding time information. The data volume in the first buffer is continuously monitored. When the data volume in the first buffer exceeds or equals a first threshold, the following switching operations are performed: stop writing new data to the first buffer; completely copy all data from the first buffer to the second buffer; clear the contents of the first buffer and reset it to its initial state. Based on this, the first buffer can resume receiving new drawing data, and the second buffer begins feature extraction and analysis of the copied data.

[0194] Therefore, this dual-buffering mechanism enables parallel processing of collected data while the user continues drawing, ensuring the continuity of data acquisition and the real-time nature of processing, and avoiding data loss or processing delays that may occur with a single buffer. Specifically, historical and full-scale structured stroke data are obtained from the feature cache pool and the buffer respectively. Incremental trajectories are determined by filtering the set of differing strokes, accurately locating the user's newly drawn content and avoiding interference from invalid data. Simultaneously, historical structured data is reused without repeated processing, reducing computational load and improving the efficiency of incremental trajectory extraction, thus laying a solid, accurate, and efficient data foundation for subsequent feature calculations and intent recognition.

[0195] In some embodiments, prior to step 2302 above, the pattern display method may further include steps 2305 to 2307 to determine structured stroke data for each individual stroke.

[0196] Step 2305: Collect multi-dimensional sensor data of the user-drawn trajectory using the sensor module; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information. In this application, the timestamp sequence information is interpreted as the timestamp corresponding to each coordinate information.

[0197] For example, multi-dimensional sensor data can be collected during the user's drawing process. Specifically, the coordinate information is a two-dimensional coordinate sequence of individual strokes {(x... ,y ),(x ,y ),...,(x ,y The sampling frequency is no less than 60Hz; the pressure sensitivity information of the screen touch position is the pressure value P={p} recorded for each sampling point. ,p ,...,p }, normalized to the [0,1] interval; the touch movement speed information can be calculated using the following formula (11) to calculate the instantaneous speed v between adjacent sampling points. i : Where, x and x Let x be the coordinates of adjacent sampling points in the x-direction, and y be the coordinates of adjacent sampling points in the x-direction. and y The coordinates of adjacent sampling points in the y-direction, where Δt is the time difference between the acquisition of adjacent sampling points. The timestamp sequence information is the timestamp T = {t} for each sampling point. ,t ,...,t } is used to analyze the drawing rhythm.

[0198] Taking coordinate information acquisition as an example, when a user draws the sun, the two-dimensional coordinate sequence of the strokes can be acquired at a sampling frequency of 60Hz. Specific sampling point information is as follows: Drawing the circular body of the sun: Draw a circle clockwise starting from the starting point. Sampling point sequence: {(200,150),(205,78),(210,77),(215,78),...,(198,152)}; Total number of sampling points: approximately 180 points (drawing time 3 seconds × 60Hz). Sunlight Ray Plotting: Draw 8 radial ray lines. Specifically, the first ray (at the 12 o'clock direction): {(250,100),(250,80),(250,60)}, the second ray (at the 3 o'clock direction): {(300,150),(320,150),(340,150)}, the third ray (at the 6 o'clock direction): {(250,200),(250,220),(250,240)}, and so on. Each ray has approximately 15-20 sampling points. Pressure Sensing Information Acquisition Example: Record the pressure value at each sampling point and normalize it to the [0,1] interval: Circular Subject Pressure Sensing Sequence: P_circle={0.3,0.35,0.4,0.45,0.5,0.48,0.42,...}; Rayleigh Pressure Sensing Sequence: P_ray1={0.6,0.65,0.7,0.68,0.6,...}; The pressure sensitivity is relatively stable (0.3-0.5) when drawing a circle, and larger (0.6-0.7) when drawing a ray. Velocity Information Calculation Example: Calculate the instantaneous velocity between adjacent sampling points, with a sampling interval Δt=1 / 60 seconds: Circular Drawing Velocity: v =√[(205-200)²+(78-150)²] / (1 / 60)=√29×60≈323.7 pixels / second. Ray drawing speed: v =√[(250-250)²+(80-100)²] / (1 / 60)=20×60=1200 pixels / second. Example of time-series information recording: Recording the timestamp sequence of each sampling point: T_circle={0.000s,0.017s,0.033s,0.050s,...,3.000s}; T_ray1={3.200s,3.217s,3.233s,...,3.500s}; T_ray2={3.700s,3.717s,3.733s,...,4.000s}.

[0199] Step 2306: Based on multi-dimensional sensor data, determine at least one independent stroke according to the user's input state when drawing a single stroke; wherein, the input state is used to characterize the user's interactive action when drawing a single stroke. In this embodiment, the input state may include a touch-down state (Down), a touch-move state (Move), and a touch-up state (Up), wherein the touch-down state is a continuous action sequence that triggers the stroke, the touch-move state is the trajectory drawing process, and the touch-up state is the stroke termination node. Specifically, the continuous trajectory can be divided into independent strokes according to the touch event's press, move, and lift states, and a unique identifier Stroke_ID is assigned to each stroke to establish an independent stroke sequence S={S ,S ,...,S }

[0200] Step 2307: Determine the structured stroke data of each independent stroke based on each independent stroke in at least one independent stroke and the multi-dimensional sensor data corresponding to each independent stroke.

[0201] In this step, each independent stroke and the corresponding multi-dimensional sensor data can be encapsulated into structured data to obtain the structured stroke data of each independent stroke. For example, ID: unique identifier of the stroke, Points: coordinate point sequence of the drawing path, Pressure: pressure sensing value sequence during the drawing process, Velocity: speed change sequence during drawing, Timestamp: timestamp sequence corresponding to each data point, BoundingBox: boundary rectangle range information of the stroke. Stroke segmentation example: According to the touch event state, the sun drawing process is divided into 9 independent strokes: Stroke_1: the circular body of the sun, Stroke_2-9: 8 rays of sunlight, each ray is an independent stroke.

[0202] Therefore, by collecting full drawing data such as touch coordinates, pressure sensitivity, speed, and timestamps through multi-dimensional sensors, the characteristics of the user's drawing trajectory can be fully captured. Based on the drawing input state, individual strokes are broken down, and then structured stroke data is constructed by combining multi-dimensional data. This not only achieves refined and structured storage of drawing trajectory data, but also provides comprehensive and accurate raw data support for subsequent incremental trajectory extraction and feature analysis, ensuring the accuracy of subsequent recognition processes.

[0203] Furthermore, step 2306 may specifically include steps 23061 and 23062.

[0204] Step 23061: Obtain the first multi-dimensional sensor data within the sliding time window from the multi-dimensional sensor data.

[0205] In this step, a dynamic sliding time window can be preset, with a corresponding window size and update rules. For example, if the current system time is 10:00 and the preset duration is 120 seconds, then the first time point can be determined as 10:02, and this first time point is used as the starting time point of the dynamic sliding time window. In this step, based on the above-mentioned dynamic sliding time window, data within the time window range is extracted from the multi-dimensional sensor data to obtain the first multi-dimensional sensor data. For example, if the window size of the dynamic sliding time window is set to 200ms, then with 10:02 as the starting time point, the multi-dimensional sensor data within 200ms after 10:02 is extracted from the second buffer as the first multi-dimensional sensor data.

[0206] Step 23062: Based on the user's input state of drawing a single stroke, generate at least one independent stroke using the second multi-dimensional sensor data in the first multi-dimensional sensor data; wherein, the second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate the independent stroke.

[0207] In this step, after obtaining the first multi-dimensional sensor data within the sliding time window, the data is segmented and identified according to the user's input states such as pressure, position, and timing when drawing a single stroke. Specifically, the portion of the first multi-dimensional sensor data not yet used to generate independent strokes is identified as the second multi-dimensional sensor data. Based on the drawing trajectory, timing changes, and input states represented by this second multi-dimensional sensor data, at least one independent stroke from the user is parsed and generated segment by segment. Through this method, unused sensor data can be continuously utilized within the sliding time window to generate independent strokes conforming to the user's drawing habits, achieving real-time segmentation and continuous generation of the drawing trajectory.

[0208] In this way, incremental sensor data is extracted by sliding time windows, and new independent strokes are generated only based on unused sensor data, accurately capturing the information of strokes newly drawn by the user and avoiding data duplication. At the same time, the real-time and progressive generation of strokes is achieved by relying on sliding windows, improving the timeliness and accuracy of stroke splitting, and providing real-time and accurate stroke data support for subsequent structured data construction and incremental trajectory extraction.

[0209] In some embodiments, the structured stroke data for each individual stroke is stored in a first buffer, and the pattern display method may further include steps 2401 to 2405 before step 2301 above.

[0210] Step 2401: If the amount of structured stroke data stored in the first buffer is greater than or equal to the third threshold, copy the structured stroke data stored in the first buffer to the second buffer and delete the structured stroke data stored in the first buffer. Here, structured stroke data refers to multi-dimensional sensor data corresponding to strokes that has been encapsulated into structured data.

[0211] In this step, a first buffer is set up to cache multi-dimensional sensor data generated during the user's drawing process in real time. This data is encapsulated into structured stroke data. When the amount of structured stroke data stored in the first buffer is greater than or equal to a preset third threshold, a buffer switching process is triggered: the accumulated structured stroke data in the first buffer is copied to the second buffer, and the original data in the first buffer is cleared, so that the first buffer is restored to a writable state and can continue to receive newly collected drawing data. The third threshold can be set by means of a data volume threshold, a stroke quantity threshold, or a time threshold. For example, it can be set to complete 3 independent strokes, collect 800 data points, or have a 200ms timeout in the dynamic sliding time window. Meeting any of these conditions will trigger this step.

[0212] Step 2402: Obtain structured stroke data for each individual stroke from the second buffer to determine at least one trajectory drawn by the user.

[0213] In this step, after obtaining structured stroke data from the first buffer in the second buffer, the structured stroke data corresponding to each independent stroke is parsed and extracted from the second buffer stroke by stroke. Based on the structured information such as the coordinate sequence, pressure, speed, and timestamp of each independent stroke, data that are sequentially continuous and belong to the same drawing action are combined to determine at least one complete trajectory currently drawn by the user. For example, data such as Points, Pressure, Velocity, and Timestamp extracted within a dynamic sliding time window can be combined to form user-drawn trajectories such as sun rays or circular outlines.

[0214] Step 2403: Based on at least one trajectory, determine the geometric feature information corresponding to each trajectory in the at least one trajectory.

[0215] In this step, for at least one trajectory determined in step 2402, geometric features are calculated for each trajectory to obtain the geometric feature information corresponding to each trajectory. The geometric feature information may include the trajectory's bounding box, Hu moment, circularity, aspect ratio, stroke length, curvature, and trajectory direction, etc., which are used to characterize the overall geometric shape and structural features of the trajectory, providing basic feature support for subsequent pattern recognition.

[0216] Step 2404: Determine the first pattern based on the geometric feature information corresponding to each trajectory.

[0217] In this step, based on the geometric feature information corresponding to each trajectory, and combined with a preset basic shape library and classification strategy, the trajectory is subjected to shape recognition and combination judgment to determine the first pattern formed by the user's drawing. For example, based on the circular outline features and radial line features, the first pattern can be determined to be a sun pattern; based on the rectangular outline features and internal line features, the first pattern can be determined to be a house pattern, etc., thus obtaining the first pattern that matches the user's drawing intention.

[0218] Step 2405: The structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern are determined as the structured trajectory data of the first pattern, and the structured trajectory data of the first pattern is stored in the feature cache pool.

[0219] In this step, after determining the first pattern, the original structured stroke data corresponding to the first pattern is generated, and the corresponding geometric feature information extracted in step 2403 is associated and encapsulated together as the structured trajectory data of the first pattern. This structured trajectory data is stored in a feature cache pool for quick retrieval in subsequent incremental drawing, similarity matching, intent prediction, or stylized rendering processes, avoiding redundant feature calculations and improving processing efficiency.

[0220] In this way, data migration and cleaning are triggered by the buffer threshold, enabling batch processing of the drawing data and avoiding data overload in a single buffer. The trajectory is extracted based on the data in the second buffer, and geometric features are calculated to determine the first pattern. Then, the structured trajectory data and geometric features are stored in the feature cache pool to achieve standardized storage and reuse of historical drawing data. This not only ensures the orderliness of data processing and the efficiency of equipment operation, but also provides accurate and callable historical feature data for subsequent incremental trajectory comparison and graphic feature integration, greatly reducing redundant calculations.

[0221] It should be noted that this embodiment also sets multiple trigger conditions to intelligently start the feature extraction process, including data volume threshold conditions, time threshold conditions, and performance threshold conditions. The data volume threshold conditions may include a sampling point number threshold, a buffer usage rate threshold, and a stroke number threshold; the time threshold conditions may include dynamic sliding time window timeout, user idle timeout, and maximum processing time limit; the performance threshold conditions may include memory usage rate, processor usage rate, and rendering frame rate limit. When any trigger condition is met, operations such as stopping data acquisition, buffer switching, starting feature extraction, resetting the acquisition environment, and recording the trigger event can be performed. Simultaneously, a dynamic sliding time window mechanism can be used to achieve real-time updates of the drawing data. The window size can be set to 200 milliseconds, with a 50-millisecond overlap area between adjacent windows, and window updates are performed at 150-millisecond intervals. Each window can accommodate a maximum of a preset number of data points, thereby ensuring the timeliness and continuity of the trajectory data and providing stable data support for real-time construction of user-drawn trajectories and feature extraction.

[0222] In some embodiments of this application, the embodiments of this application can also perform graphic completion on the incomplete pattern drawn by the user. Based on this, after step 130, the pattern display method can also include steps 2501 to 2503, as shown below.

[0223] Step 2501: Receive a second input for the intended pattern to be drawn.

[0224] In this step, the drawing intent pattern includes a cartoon sun pattern and an oil painting pattern. The user clicks or selects the sun pattern on the interface, thereby forming a second input for the drawing intent pattern.

[0225] Step 2502: In response to the second input, determine whether the first graphic drawn by the user is a complete graphic based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern.

[0226] In this step, the main graphic corresponding to the sun pattern is a circular outline, and the auxiliary graphic is radiating light rays. The system compares the user's first drawn graphic with the main and auxiliary graphics of the sun pattern to determine whether the user's first drawn graphic is a complete sun pattern. For example, if the user only draws an arc or a partial circular outline, without forming a closed circle, and without drawing any light rays, then the first graphic can be determined to be an incomplete graphic.

[0227] Step 2503: If the first graphic is incomplete, the first graphic is completed based on the main graphic and auxiliary graphic corresponding to the intended pattern, to obtain the third graphic.

[0228] In this step, since the circular outline drawn by the user is not closed and lacks ray lines, the complete circular outline can be completed based on the main graphic of the sun pattern. Radial lines are added around the circle as auxiliary graphics to complete the incomplete first graphic, ultimately resulting in a complete and regular sun pattern, i.e., the third graphic. The completed graphic can be generated using a Generative Adversarial Network (GAN) or a Variational Autoencoder (VAE).

[0229] To enhance the user's flexibility in selecting candidate objects, multiple candidate objects can be generated, increasing the user's selection space. For example, a half-circle can be completed into a full circle, a smiley face, a sun, etc. To facilitate user browsing, the completed objects can be displayed using dashed lines or semi-transparent methods, allowing users to choose whether to accept the completion or continue drawing manually.

[0230] In some embodiments of this application, after step 120, the pattern display method may further include steps 2601 to 2604, as detailed below.

[0231] Step 2601: Receive a third input for the stop drawing control or for the drawing intention pattern.

[0232] For example, it can be as follows Figure 3 As shown, when the user selects Figure 3 When generating controls with a single click, this action signifies that the user has stopped drawing; that is, the system has received a third input from the user, such as a stop-drawing input. Additionally, if the user directly clicks to select the intended drawing pattern, such as a complete sun pattern, this selection action also constitutes a third input and triggers subsequent processes. Both input methods can achieve the function of stopping the current drawing and starting the final rendering.

[0233] Step 2602, in response to the third input, generates an image frame for each stroke in the user-drawn stroke set based on the stroke set of the first graphic drawn by the user.

[0234] In this step, the user's first hand-drawn graphic, such as an incomplete sun, including an unclosed circular outline and three scattered rays, is decomposed into multiple frames of data. These multiple frames include a start frame, an intermediate frame, and an end frame, achieving a smooth transition from the original doodle to the complete drawing intention pattern. The generation of image frames can be divided into two categories. One is the generation of basic image frames, which directly extracts the original drawing state of each stroke in the user's drawing stroke set to generate a start frame, such as the user's original hand-drawn doodle frame, an unclosed arc frame, or a single ray frame. Each stroke corresponds to one original start frame, arranged according to the user's drawing order, such as drawing the arc frame first, then drawing the first ray frame, and so on. The generation of intermediate transition image frames is to generate intermediate frames through a shape interpolation algorithm to achieve a smooth transition from the start frame (original doodle) to the end frame (drawing intention pattern). The intermediate frames can be generated through a shape interpolation algorithm, i.e., Frame_t = (1-t), which can be achieved through formula (12): Here, Shape_start+t is the starting frame, i.e., the user's original drawing, and Shape_end is the ending frame, i.e., the final object. Intermediate frames refer to the progressively optimized transition graphics. t∈[0,1] is the time parameter.

[0235] Step 2603: Generate a pattern drawing video based on the image frame of each stroke and the image frame corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first or last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern.

[0236] In this step, the user draws the sun by first drawing an unclosed arc, then three rays. The corresponding image frames include: the starting frame for each stroke (e.g., the unclosed arc frame), and the individual frames corresponding to the three rays; intermediate transition frames generated based on shape interpolation algorithms (e.g., the transition frames as the arc gradually closes, and the transition frames as the number of rays gradually increases from three to eight); and the ending frame corresponding to the intended pattern (e.g., the complete sun frame). These image frames are arranged in the order in which the user draws the strokes (e.g., the starting frame of the unclosed arc, the transition frame of the closed arc, the starting frame of the first ray, the transition frames of the increasing rays, ..., the ending frame of the complete sun) to generate a pattern drawing video. The last image frame of this video is the image frame corresponding to the intended pattern, i.e., the complete sun, ensuring that the video ends with a complete and well-organized target pattern.

[0237] Step 2604: Play the image drawing video.

[0238] In this step, the video playback function is activated, and the video is played in the order of the image frames, starting from the user's original doodle frame, such as an unclosed arc or scattered rays of light, and gradually showing the intermediate transition frames, such as the arc closing and the rays of light increasing, and finally playing to the end frame, such as a complete sun pattern. This fully restores the user's hand-drawing process and presents the final optimized drawing intention pattern, achieving a coherent display of the hand-drawing process and the final effect.

[0239] In this way, in response to the input of stopping drawing or drawing an intention pattern, the system generates user stroke image frames in the drawing order and combines them with the intention pattern image frames to create a drawing video. The first or last frame is the intention pattern frame and the video is played. This not only restores the user's hand-drawing process and intuitively presents the process of generating a beautiful pattern from a doodle, but also enriches the display of the drawing results through video, improves the interactive experience and the visualization effect of the results. At the same time, no additional user operation is required, and the video can be generated and played with one click, reducing the operation threshold.

[0240] In some embodiments of this application, after step 120, the drawing intention pattern includes a drawing intention pattern of a first type of painting style and a drawing intention of a second type of painting style. The pattern display method may further include step 2701, including: The drawing intent pattern of the first type of painting style is displayed according to the first display size, and the drawing intent pattern of the second type of painting style is displayed according to the second display size; wherein the first display size is larger than the second display size; and the confidence level of the first type of painting style is greater than the confidence level of the second type of painting style.

[0241] In this step, the top three painting style objects with the highest confidence scores are first selected from multiple conversation style objects as candidate painting styles to be displayed. These three painting style objects must have certain differences to avoid overly similar styles. The selected top three painting style objects can be oil painting, anime / manga, and cartoon styles. Then, these three candidate painting styles can be divided into two categories: the painting style with the highest confidence score (Top-1) is determined as the first category, and the two painting styles with the next highest confidence scores (Top-2 and Top-3) are determined as the second category. Based on the difference in confidence scores between the two categories of painting styles, different display sizes are used in the style selection area of ​​the drawing interface, such as... Figure 4The central area 303 displays images horizontally for easy browsing and selection. It shows the drawing intent patterns for the first category of painting styles at a first display size, and the drawing intent patterns for the second category of painting styles at a second display size, with the first display size pre-set to be larger than the second. The first display size is set to 80×80 pixels to prominently display the drawing intent patterns for the highest-confidence painting styles, such as cartoon styles, attracting user attention. The second display size is set to 60×60 pixels to display the drawing intent patterns for the next two-thirds of the highest-confidence painting styles, such as oil painting and anime styles, at a medium size. This ensures clear visibility while differentiating the display from the first category of painting styles, achieving a focused and hierarchical display effect and improving user selection efficiency.

[0242] Furthermore, users can click on any painting style object and share it with a contact selected by the user. Also, the number of painting style objects can be more, such as more than three. In this case, area 303 in this embodiment should support swiping left and right to view more candidate results. It should be noted that at least two embodiments in this application can be used in combination. For example, continuing to refer to... Figure 4 The canvas in the middle contains the user's original doodles, such as... Figure 4 The oil painting shown is a sun doodle, which can be used as a complete image. The middle image is the one with the highest confidence level, which best matches the cartoon smiley face sun. The image on the left is the second most confident image, which is a watercolor sun and clouds.

[0243] In this way, different display sizes are matched to different styles of intent patterns based on the confidence level. High-confidence patterns are displayed in larger sizes, while low-confidence patterns are displayed in smaller sizes, allowing users to quickly focus on the best matching results, improving the visual recognition and selection efficiency of intent patterns, while clearly distinguishing candidate solutions with different confidence levels, taking into account both selection diversity and ease of operation.

[0244] The pattern display method provided in this application can be executed by a pattern display device. This application uses a pattern display device to perform the display as an example to illustrate the apparatus of the pattern display method provided in this application.

[0245] This application also provides a pattern display device. (Specifically combined with...) Figure 8 Please provide a detailed explanation.

[0246] Figure 8 This is a schematic diagram of the structure of a pattern display device provided for some embodiments of this application.

[0247] like Figure 8As shown, the pattern display device 80 can be applied to electronic devices, and the pattern display device 80 may specifically include: The receiving module 801 is used to receive a first input when the first pattern is displayed, wherein the first input is the input of the user drawing the trajectory; Display module 802 is configured to display at least one drawing intent pattern in response to a first input; Wherein, at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and a first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory including the trajectory drawn by the user.

[0248] The pattern display device 80 in the embodiments of this application will be described in detail below.

[0249] In some embodiments of this application, the pattern display device 80 may further include a determining module for determining geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory; Based on the geometric increment feature information and the first pattern, determine the graphic feature information of the second pattern; Based on the graphic feature information and reference information of the second pattern, the intended pattern for drawing is determined; The graphic feature information includes geometric feature information and structural feature information. The structural feature information includes the positional information of the shape corresponding to the geometric feature information.

[0250] In this way, the geometric incremental features of the incremental trajectory can be accurately extracted, and combined with the first pattern to integrate complete graphic features including geometry and position structure. With reference information, the drawing intent can be determined from multiple dimensions, which not only reduces redundant calculations and improves recognition efficiency, but also makes the matching of intent patterns more accurate and meets the actual drawing needs of users.

[0251] In some embodiments of this application, the pattern display device 80 may further include a filtering module for filtering reference patterns that match the graphic feature information of the second pattern from a database based on the graphic feature information of the second pattern. The pattern display device 80 may further include a determining module for determining a drawing intention pattern from a reference pattern based on the semantic context information of the reference information; The semantic context includes intent keywords and intent enhancement feature information; intent keywords are the keywords in the reference information that indicate the user's intent to draw a pattern, and intent enhancement feature information is the information that describes the pattern features indicated by the intent keywords.

[0252] In this way, reference patterns can be accurately screened from the database based on graphic features to narrow down the matching range. Then, the semantic-level secondary screening is performed by combining the intent keywords of the reference information and the intent enhancement information of the pattern features to determine the drawing intent pattern. The dual screening can not only improve the pattern matching efficiency, but also make the results more in line with the user's actual drawing intent, which can greatly improve the recognition accuracy.

[0253] In some embodiments of this application, the pattern display device 80 may further include a determining module, used to determine the main graphic and auxiliary graphic in the second pattern based on the graphic feature information of the second pattern, when the database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database. The pattern display device 80 may also include a filtering module for filtering a first type of reference pattern corresponding to the main pattern from a reference pattern database; The filtering module can also be used to filter reference patterns that match the first type of reference pattern from a database of reference patterns corresponding to the first type of reference pattern, based on the auxiliary graphic.

[0254] In this way, by splitting the main graphic and auxiliary graphic, and combining the hierarchical reference graphic library and reference pattern library for step-by-step filtering, the main graphic is first determined to narrow the search range, and then the auxiliary graphic is used to accurately match the pattern, which greatly improves the database retrieval efficiency. At the same time, the pattern matching is made to better fit the graphic feature details of the second pattern, thereby improving the matching accuracy.

[0255] In some embodiments of this application, the pattern display device may further include a determination module, configured to determine a user drawing intent score value for each of the at least two reference patterns based on the semantic context information of the reference information, when the number of reference patterns is at least two; the user drawing intent score value is used to characterize the degree of similarity between the reference pattern and the semantic context information of the reference information. At least one reference pattern whose user drawing intent score is greater than or equal to a first threshold is identified as at least one drawing intent pattern.

[0256] In this way, by calculating the intent score for each reference pattern through semantic context, the similarity between the pattern and the user's drawing intent is quantified, and then the matching drawing intent pattern is filtered out by the score threshold. This not only realizes the quantitative judgment of intent matching and improves the objectivity and accuracy of the filtering, but also efficiently filters low-matching patterns, ensuring that the drawing intent pattern is highly consistent with the user's expectations.

[0257] In some embodiments of this application, the pattern display device 80 may further include an extraction module for extracting M word units from N text records, where M is an integer greater than or equal to 2, when the reference information includes N text records, where N is an integer greater than or equal to 2. The pattern display device 80 may further include a determining module for determining the word weight of a word based on a first frequency, a first quantity, and a second quantity; the first frequency is the frequency of the word appearing in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records including the word. The determination module can also be used to identify words with weights greater than or equal to a first threshold as intent keywords for user-drawn intent patterns in the reference information.

[0258] In this way, by extracting lexical units from multiple text records, combining the frequency of lexical unit occurrence, the total number of texts, and the quantification of texts containing the lexical unit to calculate the lexical unit weight, and then filtering intent keywords according to thresholds, the core semantics in the reference information are accurately extracted, providing accurate semantic basis for subsequent pattern intent matching and improving the matching degree between the drawn intent pattern and the user's semantic needs.

[0259] In some embodiments of this application, the pattern display device 80 may further include an acquisition module for inputting intent keywords into a word vector model and acquiring the intent word vectors of the intent keywords output by the word vector model; The pattern display device 80 may also include a processing module for inputting the intent word vector and the lexical weights of the intent keywords into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keywords output by the Chinese pre-trained model. The processing module can also be used to perform contextual enhancement processing on the semantic enhancement vector through a time decay factor to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information.

[0260] In this way, the semantic expression of intent keywords is strengthened layer by layer through word vector models and Chinese pre-trained models. Combined with time decay factors, the semantic vectors are enhanced with context, and recent reference information is given higher weight. The intent enhancement feature information that fits the latest intent of the user is accurately extracted, providing more accurate and timely semantic basis for subsequent pattern semantic matching, and greatly improving the matching accuracy between the drawn intent pattern and the user's actual semantic needs.

[0261] In some embodiments of this application, the pattern display device 80 may further include an adjustment module for adjusting the display size of the incremental trajectory to a reference size, thereby obtaining two-dimensional spatial feature information of the incremental trajectory at the reference size. The pattern display device 80 may further include a determining module for determining a spatiotemporal feature map of the incremental trajectory based on two-dimensional spatial feature information and temporal feature information of the incremental trajectory. The determination module can also be used to determine two-dimensional dynamic feature information based on multi-dimensional sensor data corresponding to the incremental trajectory; The pattern display device 80 may also include a processing module for inputting spatiotemporal feature maps and two-dimensional dynamic feature information into a convolutional neural network, and obtaining geometric increment feature information output by the convolutional neural network corresponding to the increment trajectory.

[0262] In this way, the incremental trajectory is standardized and its size is adjusted, and the spatiotemporal features are integrated with the dynamic features of multi-dimensional sensors. Then, geometric incremental features are extracted through a convolutional neural network. This not only ensures the standardization and comprehensiveness of feature extraction, but also enables in-depth mining of incremental features with the help of neural networks. This makes the extracted geometric incremental features more accurate and more in line with the actual features of the incremental trajectory, laying a precise feature foundation for subsequent graphic feature integration and pattern matching.

[0263] In some embodiments of this application, the pattern display device 80 may further include an acquisition module for acquiring structured trajectory data of a first pattern from a feature cache pool; the structured trajectory data includes historical structured stroke data, and the historical structured stroke data includes a first stroke set; The acquisition module can also be used to acquire the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes. The pattern display device 80 may also include a filtering module for filtering a third set of strokes from the second set of strokes that does not overlap with the first set of strokes; The pattern display device 80 may also include a determination module for determining an incremental trajectory based on structured stroke data corresponding to each individual stroke in the third stroke set.

[0264] In this way, historical and full-scale structured stroke data are obtained from the feature cache pool and buffer respectively. By filtering the set of differences in strokes, the incremental trajectory is determined, accurately locating the user's newly drawn content and avoiding interference from invalid data. At the same time, historical structured data is reused without repeated processing, reducing the amount of computation and improving the efficiency of incremental trajectory extraction, thus laying a solid, accurate, and efficient data foundation for subsequent feature calculation and intent recognition.

[0265] In some embodiments of this application, the pattern display device 80 may further include a data acquisition module for acquiring multi-dimensional sensor data of the user-drawn trajectory through a sensor module; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information; The pattern display device 80 may further include a determining module for determining at least one independent stroke based on multi-dimensional sensor data and according to the user's input state when drawing a single stroke; wherein the input state is used to characterize the state of the user's interactive action when drawing a single stroke. The determination module can also be used to determine the structured stroke data of each independent stroke based on each independent stroke in at least one independent stroke and the multi-dimensional sensor data corresponding to each independent stroke.

[0266] In this way, by collecting full drawing data such as touch coordinates, pressure sensitivity, speed, and timestamps through multi-dimensional sensors, the user's drawing trajectory features can be fully captured. Based on the drawing input state, individual strokes are broken down, and then structured stroke data is constructed by combining multi-dimensional data. This not only achieves refined and structured storage of drawing trajectory data, but also provides comprehensive and accurate raw data support for subsequent incremental trajectory extraction and feature analysis, ensuring the accuracy of subsequent recognition processes.

[0267] In some embodiments of this application, the pattern display device 80 may further include an acquisition module for acquiring first multi-dimensional sensor data within a sliding time window from multi-dimensional sensor data; The pattern display device 80 may also include a generation module for generating at least one independent stroke based on second multi-dimensional sensor data in the first multi-dimensional sensor data, according to the user's input state of drawing a single stroke. The second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate independent strokes.

[0268] In this way, incremental sensor data is extracted by sliding time windows, and new independent strokes are generated only based on unused sensor data, accurately capturing the information of strokes newly drawn by the user and avoiding data duplication. At the same time, the real-time and progressive generation of strokes is achieved by relying on sliding windows, improving the timeliness and accuracy of stroke splitting, and providing real-time and accurate stroke data support for subsequent structured data construction and incremental trajectory extraction.

[0269] In some embodiments of this application, the pattern display device 80 may further include a storage module, which is used to copy the structured stroke data stored in the first buffer to a second buffer and delete the structured stroke data stored in the first buffer when the structured stroke data of each independent stroke exists in a first buffer and the amount of structured stroke data stored in the first buffer is greater than or equal to a third threshold. The pattern display device 80 may further include a determination module for obtaining structured stroke data of each individual stroke from the second buffer and determining at least one trajectory drawn by the user. The determination module can also be used to determine the geometric feature information corresponding to each trajectory in at least one trajectory, based on at least one trajectory. The determination module can also be used to determine the first pattern based on the geometric feature information corresponding to each trajectory; The storage module can also be used to determine the structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern as the structured trajectory data of the first pattern, and store the structured trajectory data of the first pattern in the feature cache pool.

[0270] In this way, data migration and cleaning are triggered by the buffer threshold, enabling batch processing of the drawing data and avoiding data overload in a single buffer. The trajectory is extracted based on the data in the second buffer, and geometric features are calculated to determine the first pattern. Then, the structured trajectory data and geometric features are stored in the feature cache pool to achieve standardized storage and reuse of historical drawing data. This not only ensures the orderliness of data processing and the efficiency of equipment operation, but also provides accurate and callable historical feature data for subsequent incremental trajectory comparison and graphic feature integration, greatly reducing redundant calculations.

[0271] In some embodiments of this application, the receiving module 801 may also be used to receive a second input to the intended drawing pattern; The pattern display device 80 may further include a determination module for determining, in response to a second input, whether the first graphic drawn by the user is a complete graphic based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern; The pattern display device 80 may also include a completion module, which, when the first graphic is incomplete, completes the first graphic based on the main graphic and auxiliary graphic corresponding to the intended pattern to obtain a third graphic.

[0272] In this way, the system receives and responds to secondary input of the intended drawing pattern, first determines the integrity of the graphic, and then, for incomplete graphics, intelligently completes the graphic based on the main and auxiliary graphics of the intended drawing pattern to obtain a new graphic. This not only achieves accurate determination of graphic integrity, but also can selectively complete graphic details based on matching intention features, so that the completed graphic fits the user's drawing intention, improves graphic expression, enhances the integrity and accuracy of graphic drawing, and reduces the user's manual completion operation cost.

[0273] In some embodiments of this application, the receiving module 801 may also be used to receive a third input to the stop drawing control or to the drawing intention pattern; The pattern display device 80 may further include a generation module for generating an image frame of each stroke in the user-drawn stroke set in response to a third input, based on the stroke set of the user-drawn first graphic. The generation module can also be used to generate a pattern drawing video based on the image frame of each stroke and the image frame corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first or last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern; The pattern display device 80 may also include a playback module for playing image drawing videos.

[0274] In this way, in response to the input of stopping drawing or drawing an intention pattern, the system generates user stroke image frames in the drawing order and combines them with the intention pattern image frames to create a drawing video. The first or last frame is the intention pattern frame and the video is played. This not only restores the user's hand-drawing process and intuitively presents the process of generating a beautiful pattern from a doodle, but also enriches the display of the drawing results through video, improves the interactive experience and the visualization effect of the results. At the same time, no additional user operation is required, and the video can be generated and played with one click, reducing the operation threshold.

[0275] In some embodiments of this application, the display module 802 may also be used to display the first type of painting style painting intention pattern according to a first display size and the second type of painting style painting intention pattern according to a second display size when the painting intention pattern includes a first type of painting style painting intention pattern and a second type of painting style painting intention pattern. Among them, the first display size is larger than the second display size; the confidence level of the first type of painting style is greater than that of the second type of painting style.

[0276] In this way, different display sizes are matched to different styles of intent patterns based on the confidence level. High-confidence patterns are displayed in larger sizes, while low-confidence patterns are displayed in smaller sizes, allowing users to quickly focus on the best matching results, improving the visual recognition and selection efficiency of intent patterns, while clearly distinguishing candidate solutions with different confidence levels, taking into account both selection diversity and ease of operation.

[0277] The pattern display device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0278] The pattern display device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0279] The device coordination apparatus provided in this application embodiment can achieve... Figures 1 to 7 The various processes implemented in the illustrated pattern display method embodiment achieve the same technical effect, and will not be described again here to avoid repetition.

[0280] Based on this, the pattern display device provided in this application, when displaying a first pattern, receives a first input, which is an input of a user drawing a trajectory; in response to the first input, it displays at least one drawing intention pattern; wherein, the at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern is composed of an incremental trajectory corresponding to the first input and the first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory includes the user drawing trajectory. In this way, by obtaining the corresponding incremental trajectory when the user makes the first input, and combining the incremental trajectory with the first pattern to form a second pattern for recognition, the recognition process does not rely on complete drawing, significantly shortening the recognition response time and greatly improving the real-time performance of recognizing user input content. Moreover, feature extraction and calculation are performed only on the incremental trajectory portion, eliminating the need for repeated calculation of features related to the first pattern, effectively reducing computational resource consumption and lowering the device's operating load. Furthermore, displaying at least one drawing intention pattern as a candidate object in real time after responding to the first input facilitates the user's immediate confirmation of the desired pattern or continued adjustment of the drawing trajectory, avoiding waiting for complete input before making modifications, further improving the efficiency of user input operations. At the same time, semantic features are constructed by combining user chat history, notes and other reference information and fused with the graphic features of the second pattern. Based on multimodal features, the user's drawing intention is accurately predicted, making the candidate drawing intention pattern more in line with the user's actual expectations, effectively reducing misidentification and secondary modification operations, and significantly improving the accuracy of recognizing user input content.

[0281] Optional, such as Figure 9 As shown, this application embodiment also provides an electronic device 90, including a processor 901 and a memory 902. The memory 902 stores a program or instructions that can run on the processor 901. When the program or instructions are executed by the processor 901, they implement the various steps of the above-described pattern display method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0282] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0283] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided for some embodiments of this application.

[0284] The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 10010, interface unit 1008, memory 1009, processor 1010, etc.

[0285] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0286] In this embodiment, the user input unit 10010 is configured to receive a first input, which is an input of a user drawing a trajectory, when a first pattern is displayed. The display unit 1006 is configured to display at least one drawing intention pattern in response to the first input; wherein the at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and the first pattern; the reference information includes at least one of the following: user chat history, note information, and the incremental trajectory including the trajectory drawn by the user.

[0287] Therefore, by acquiring the corresponding incremental trajectory when the user makes the first input, and combining the incremental trajectory with the first pattern to form the second pattern for recognition, the recognition process does not rely on the complete drawing, significantly shortening the recognition response time and greatly improving the real-time performance of recognizing user input. Moreover, feature extraction and calculation are performed only on the incremental trajectory, eliminating the need to repeatedly calculate the relevant features of the first pattern, effectively reducing computational resource consumption and lowering the device's operating load. Furthermore, after responding to the first input, at least one drawing intent pattern is displayed in real time as a candidate, allowing the user to immediately confirm the desired pattern or continue adjusting the drawing trajectory, avoiding waiting for the complete input before making modifications, further improving the efficiency of user input operations. At the same time, semantic features are constructed by combining user chat history, notes, and other reference information and fused with the graphic features of the second pattern. Based on multimodal features, the user's drawing intent is accurately predicted, making the candidate drawing intent patterns more in line with the user's actual expectations, effectively reducing misidentification and secondary modification operations, and significantly improving the accuracy of recognizing user input.

[0288] The electronic device 1000 will be described in detail below.

[0289] In some embodiments of this application, the processor 1010 is configured to determine geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory; Based on the geometric increment feature information and the first pattern, determine the graphic feature information of the second pattern; Based on the graphic feature information and reference information of the second pattern, the intended pattern for drawing is determined; The graphic feature information includes geometric feature information and structural feature information. The structural feature information includes the positional information of the shape corresponding to the geometric feature information.

[0290] In this way, the geometric incremental features of the incremental trajectory can be accurately extracted, and combined with the first pattern to integrate complete graphic features including geometry and position structure. With reference information, the drawing intent can be determined from multiple dimensions, which not only reduces redundant calculations and improves recognition efficiency, but also makes the matching of intent patterns more accurate and meets the actual drawing needs of users.

[0291] In some embodiments of this application, the processor 1010 is configured to filter reference patterns that match the graphic feature information of the second pattern from a database based on the graphic feature information of the second pattern. Based on the semantic context information of the reference information, determine the intended drawing pattern from the reference pattern; The semantic context includes intent keywords and intent enhancement feature information; intent keywords are the keywords in the reference information that indicate the user's intent to draw a pattern, and intent enhancement feature information is the information that describes the pattern features indicated by the intent keywords.

[0292] In this way, reference patterns can be accurately screened from the database based on graphic features to narrow down the matching range. Then, the semantic-level secondary screening is performed by combining the intent keywords of the reference information and the intent enhancement information of the pattern features to determine the drawing intent pattern. The dual screening can not only improve the pattern matching efficiency, but also make the results more in line with the user's actual drawing intent, which can greatly improve the recognition accuracy.

[0293] In some embodiments of this application, the processor 1010 is configured to determine the main graphic and auxiliary graphic in the second pattern based on the graphic feature information of the second pattern, when the database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database. From the reference image database, select the first category of reference images that correspond to the main image; Based on the auxiliary graphics, reference patterns that match the first type of reference graphics are selected from the reference pattern database corresponding to the first type of reference graphics.

[0294] In this way, by splitting the main graphic and auxiliary graphic, and combining the hierarchical reference graphic library and reference pattern library for step-by-step filtering, the main graphic is first determined to narrow the search range, and then the auxiliary graphic is used to accurately match the pattern, which greatly improves the database retrieval efficiency. At the same time, the pattern matching is made to better fit the graphic feature details of the second pattern, thereby improving the matching accuracy.

[0295] In some embodiments of this application, the processor 1010 is configured to determine a user drawing intent score value for each of the at least two reference patterns based on the semantic context information of the reference information, when the number of reference patterns is at least two; the user drawing intent score value is used to characterize the degree of similarity between the reference pattern and the semantic context information of the reference information. At least one reference pattern whose user drawing intent score is greater than or equal to a first threshold is identified as at least one drawing intent pattern.

[0296] In this way, by calculating the intent score for each reference pattern through semantic context, the similarity between the pattern and the user's drawing intent is quantified, and then the matching drawing intent pattern is filtered out by the score threshold. This not only realizes the quantitative judgment of intent matching and improves the objectivity and accuracy of the filtering, but also efficiently filters low-matching patterns, ensuring that the drawing intent pattern is highly consistent with the user's expectations.

[0297] In some embodiments of this application, processor 1010 is configured to extract M tokens from N text records, where M is an integer greater than or equal to 2, when the reference information includes N text records, where N is an integer greater than or equal to 2. The lexical weight is determined based on the first frequency, the first quantity, and the second quantity; the first frequency is the frequency of the lexical occurrence in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records that include the lexical. Terminators with a weight greater than or equal to the first threshold are identified as intent keywords for the user's intent pattern in the reference information.

[0298] In this way, by extracting lexical units from multiple text records, combining the frequency of lexical unit occurrence, the total number of texts, and the quantification of texts containing the lexical unit to calculate the lexical unit weight, and then filtering intent keywords according to thresholds, the core semantics in the reference information are accurately extracted, providing accurate semantic basis for subsequent pattern intent matching and improving the matching degree between the drawn intent pattern and the user's semantic needs.

[0299] In some embodiments of this application, the processor 1010 is configured to input intent keywords into a word vector model and obtain intent word vectors of intent keywords output by the word vector model; Input the intent word vector and the lexical weights of the intent keywords into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keywords output by the Chinese pre-trained model; By applying a time decay factor, the semantic enhancement vector is subjected to contextual enhancement processing to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information.

[0300] In this way, the semantic expression of intent keywords is strengthened layer by layer through word vector models and Chinese pre-trained models. Combined with time decay factors, the semantic vectors are enhanced with context, and recent reference information is given higher weight. The intent enhancement feature information that fits the latest intent of the user is accurately extracted, providing more accurate and timely semantic basis for subsequent pattern semantic matching, and greatly improving the matching accuracy between the drawn intent pattern and the user's actual semantic needs.

[0301] In some embodiments of this application, the processor 1010 is used to adjust the display size of the incremental trajectory to a reference size to obtain two-dimensional spatial feature information of the incremental trajectory at the reference size; Based on two-dimensional spatial feature information and temporal feature information of incremental trajectory, the spatiotemporal feature map of incremental trajectory is determined; Based on multi-dimensional sensor data corresponding to the incremental trajectory, two-dimensional dynamic feature information is determined; The spatiotemporal feature map and two-dimensional dynamic feature information are input into the convolutional neural network, and the geometric increment feature information corresponding to the increment trajectory is obtained from the output of the convolutional neural network.

[0302] In this way, the incremental trajectory is standardized and its size is adjusted, and the spatiotemporal features are integrated with the dynamic features of multi-dimensional sensors. Then, geometric incremental features are extracted through a convolutional neural network. This not only ensures the standardization and comprehensiveness of feature extraction, but also enables in-depth mining of incremental features with the help of neural networks. This makes the extracted geometric incremental features more accurate and more in line with the actual features of the incremental trajectory, laying a precise feature foundation for subsequent graphic feature integration and pattern matching.

[0303] In some embodiments of this application, the processor 1010 is configured to obtain structured trajectory data of a first pattern from a feature cache pool; the structured trajectory data includes historical structured stroke data, and the historical structured stroke data includes a first stroke set; Obtain the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes. From the second set of strokes, select the third set of strokes that does not overlap with the first set of strokes; The incremental trajectory is determined based on the structured stroke data corresponding to each independent stroke in the third stroke set.

[0304] In this way, historical and full-scale structured stroke data are obtained from the feature cache pool and buffer respectively. By filtering the set of differences in strokes, the incremental trajectory is determined, accurately locating the user's newly drawn content and avoiding interference from invalid data. At the same time, historical structured data is reused without repeated processing, reducing the amount of computation and improving the efficiency of incremental trajectory extraction, thus laying a solid, accurate, and efficient data foundation for subsequent feature calculation and intent recognition.

[0305] In some embodiments of this application, the processor 1010 is configured to collect multi-dimensional sensor data of a user-drawn trajectory via a sensor module; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information; Based on multi-dimensional sensor data, at least one independent stroke is determined according to the user's input state when drawing a single stroke; wherein, the input state is used to characterize the state of the user's interactive action when drawing a single stroke. The structured stroke data of each independent stroke is determined based on each independent stroke in at least one independent stroke and the multi-dimensional sensor data corresponding to each independent stroke.

[0306] In this way, by collecting full drawing data such as touch coordinates, pressure sensitivity, speed, and timestamps through multi-dimensional sensors, the user's drawing trajectory features can be fully captured. Based on the drawing input state, individual strokes are broken down, and then structured stroke data is constructed by combining multi-dimensional data. This not only achieves refined and structured storage of drawing trajectory data, but also provides comprehensive and accurate raw data support for subsequent incremental trajectory extraction and feature analysis, ensuring the accuracy of subsequent recognition processes.

[0307] In some embodiments of this application, processor 1010 is configured to acquire first multi-dimensional sensor data within a sliding time window from multi-dimensional sensor data; Based on the user's input state of drawing a single stroke, and using the second multi-dimensional sensor data in the first multi-dimensional sensor data, at least one independent stroke is generated. The second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate independent strokes.

[0308] In this way, incremental sensor data is extracted by sliding time windows, and new independent strokes are generated only based on unused sensor data, accurately capturing the information of strokes newly drawn by the user and avoiding data duplication. At the same time, the real-time and progressive generation of strokes is achieved by relying on sliding windows, improving the timeliness and accuracy of stroke splitting, and providing real-time and accurate stroke data support for subsequent structured data construction and incremental trajectory extraction.

[0309] In some embodiments of this application, the memory 1009 is used to copy the structured stroke data stored in the first buffer to a second buffer and delete the structured stroke data stored in the first buffer when the structured stroke data of each independent stroke exists in a first buffer and the amount of structured stroke data stored in the first buffer is greater than or equal to a third threshold. Processor 1010 is used to obtain structured stroke data for each individual stroke from the second buffer and determine at least one trajectory drawn by the user; The processor 1010 can also be used to determine the geometric feature information corresponding to each trajectory in at least one trajectory, based on at least one trajectory. The processor 1010 can also be used to determine a first pattern based on the geometric feature information corresponding to each trajectory; The memory 1009 can also be used to determine the structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern as the structured trajectory data of the first pattern, and store the structured trajectory data of the first pattern in the feature cache pool.

[0310] In this way, data migration and cleaning are triggered by the buffer threshold, enabling batch processing of the drawing data and avoiding data overload in a single buffer. The trajectory is extracted based on the data in the second buffer, and geometric features are calculated to determine the first pattern. Then, the structured trajectory data and geometric features are stored in the feature cache pool to achieve standardized storage and reuse of historical drawing data. This not only ensures the orderliness of data processing and the efficiency of equipment operation, but also provides accurate and callable historical feature data for subsequent incremental trajectory comparison and graphic feature integration, greatly reducing redundant calculations.

[0311] In some embodiments of this application, the user input unit 10010 may also be used to receive a second input for drawing an intention pattern; The processor 1010 is configured to respond to the second input and determine whether the first graphic drawn by the user is a complete graphic based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern. The processor 1010 is used to complete the first graphic based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern when the first graphic is incomplete, so as to obtain the third graphic.

[0312] In this way, the system receives and responds to secondary input of the intended drawing pattern, first determines the integrity of the graphic, and then, for incomplete graphics, intelligently completes the graphic based on the main and auxiliary graphics of the intended drawing pattern to obtain a new graphic. This not only achieves accurate determination of graphic integrity, but also can selectively complete graphic details based on matching intention features, so that the completed graphic fits the user's drawing intention, improves graphic expression, enhances the integrity and accuracy of graphic drawing, and reduces the user's manual completion operation cost.

[0313] In some embodiments of this application, the user input unit 10010 may also be used to receive a third input for a stop drawing control or for a drawing intention pattern; Processor 1010 is configured to, in response to a third input, generate an image frame for each stroke in the user-drawn stroke set based on the stroke set of the first graphic drawn by the user. The processor 1010 can also be used to generate a pattern drawing video based on the image frame of each stroke and the image frame corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first image frame or the last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern; Processor 1010 is used for playing images and drawing videos.

[0314] In this way, in response to the input of stopping drawing or drawing an intention pattern, the system generates user stroke image frames in the drawing order and combines them with the intention pattern image frames to create a drawing video. The first or last frame is the intention pattern frame and the video is played. This not only restores the user's hand-drawing process and intuitively presents the process of generating a beautiful pattern from a doodle, but also enriches the display of the drawing results through video, improves the interactive experience and the visualization effect of the results. At the same time, no additional user operation is required, and the video can be generated and played with one click, reducing the operation threshold.

[0315] In some embodiments of this application, the display unit 1006 may also be used to display the drawing intention pattern of the first type of painting style according to a first display size, and to display the drawing intention pattern of the second type of painting style according to a second display size, when the drawing intention pattern includes a drawing intention pattern of the first type of painting style and a drawing intention of the second type of painting style. Among them, the first display size is larger than the second display size; the confidence level of the first type of painting style is greater than that of the second type of painting style.

[0316] In this way, different display sizes are matched to different styles of intent patterns based on the confidence level. High-confidence patterns are displayed in larger sizes, while low-confidence patterns are displayed in smaller sizes, allowing users to quickly focus on the best matching results, improving the visual recognition and selection efficiency of intent patterns, while clearly distinguishing candidate solutions with different confidence levels, taking into account both selection diversity and ease of operation.

[0317] It should be understood that the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image information of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1006 may include a display panel, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 10010 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch display. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume display buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0318] The memory 1009 can be used to store software programs and various information. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing information. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0319] Processor 1010 may include one or more processing units; optionally, processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless display signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1010.

[0320] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described pattern display method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0321] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0322] In addition, this application embodiment provides another chip, which includes a processor and a display interface. The display interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described pattern display method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0323] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0324] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described pattern display method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0325] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0326] Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0327] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0328] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A pattern display method characterized by, include: While displaying the first pattern, receive the first input; The first input is the user's input for drawing the trajectory; In response to the first input, at least one drawing intent pattern is displayed; Wherein, the at least one drawing intention pattern is determined based on a second pattern and reference information, the second pattern being composed of an incremental trajectory corresponding to the first input and the first pattern; the reference information includes at least one of the following: user chat history, note information; the incremental trajectory includes a trajectory drawn by the user.

2. The method according to claim 1, characterized in that, Before displaying at least one drawing intent pattern, the method further includes: Based on the incremental trajectory, determine the geometric incremental feature information corresponding to the incremental trajectory; Based on the geometric increment feature information and the first pattern, the graphic feature information of the second pattern is determined; Based on the graphic feature information of the second pattern and the reference information, the intended drawing pattern is determined; The graphic feature information includes geometric feature information and structural feature information, and the structural feature information includes the position information of the shape corresponding to the geometric feature information.

3. The method according to claim 2, characterized in that, Determining the intended drawing pattern based on the graphic feature information of the second pattern and the reference information includes: Based on the graphic feature information of the second pattern, reference patterns that match the graphic feature information of the second pattern are selected from the database. Based on the semantic context information of the reference information, the drawing intention pattern is determined from the reference pattern; The semantic context includes intent keywords and intent enhancement feature information; the intent keywords are keywords in the reference information indicating the user's intent to draw a pattern, and the intent enhancement feature information is information describing the pattern features indicated by the intent keywords.

4. The method according to claim 3, characterized in that, The database includes a reference graphic database and a reference pattern database corresponding to each type of reference graphic in the reference graphic database; the step of filtering reference patterns from the database that match the graphic feature information of the second pattern, based on the graphic feature information of the second pattern, includes: Based on the graphic feature information of the second pattern, determine the main graphic and auxiliary graphic in the second pattern; From the reference image database, select a first category of reference images that correspond to the main image; Based on the auxiliary graphic, reference patterns that match the first type of reference graphic are selected from the reference pattern database corresponding to the first type of reference graphic.

5. The method according to claim 3 or 4, characterized in that, The number of reference patterns is at least two; determining the drawing intention pattern from the reference patterns based on the semantic context information of the reference information includes: Based on the semantic context information of the reference information, a user drawing intent score value is determined for each of at least two reference patterns; the user drawing intent score value is used to characterize the degree of similarity between the reference pattern and the semantic context information of the reference information. At least one reference pattern whose user drawing intent score is greater than or equal to a first threshold is identified as at least one drawing intent pattern.

6. The method according to claim 3, characterized in that, The reference information includes N text records, where N is an integer greater than or equal to 2; before determining the drawing intention pattern from the reference pattern based on the semantic context information of the reference information, the method further includes: Extract M tokens from the N text records, where M is an integer greater than or equal to 2; The lexical weight is determined based on a first frequency, a first quantity, and a second quantity; the first frequency is the frequency at which the lexical appears in the reference information; the first quantity is the number of text records; and the second quantity is the number of text records including the lexical. The lexical units with a weight greater than or equal to the first threshold are identified as the intent keywords for the user to draw the intent pattern in the reference information.

7. The method according to claim 6, characterized in that, Before determining the drawing intent pattern from the reference pattern based on the semantic context information of the reference information, the method further includes: Input the intent keyword into the word vector model and obtain the intent word vector of the intent keyword output by the word vector model; Input the intent word vector and the lexical weights of the intent keywords into the Chinese pre-trained model to obtain the semantic enhancement vector of the intent keywords output by the Chinese pre-trained model; By applying a time decay factor, the semantic enhancement vector is subjected to context enhancement processing to obtain intent enhancement feature information; The time decay factor is determined by the time difference and the decay coefficient; the time difference is the difference between the acquisition time of the semantic enhancement vector and the acquisition time of the reference information.

8. The method according to claim 2, characterized in that, The step of determining the geometric incremental feature information corresponding to the incremental trajectory based on the incremental trajectory includes: The display size of the incremental trajectory is adjusted to the reference size to obtain the two-dimensional spatial feature information of the incremental trajectory under the reference size; Based on the two-dimensional spatial feature information and the temporal feature information of the incremental trajectory, the spatiotemporal feature map of the incremental trajectory is determined; Based on the multi-dimensional sensor data corresponding to the incremental trajectory, two-dimensional dynamic feature information is determined; The spatiotemporal feature map and the two-dimensional dynamic feature information are input into a convolutional neural network, and the geometric incremental feature information output by the convolutional neural network corresponding to the incremental trajectory is obtained.

9. The method according to claim 1, characterized in that, Before displaying at least one drawing intent pattern, the method further includes: The structured trajectory data of the first pattern is obtained from the feature cache pool; the structured trajectory data includes historical structured stroke data, and the historical structured stroke data includes the first stroke set. Obtain the full set of structured stroke data from the first buffer; the full set of structured stroke data includes the second set of strokes. From the second set of strokes, select a third set of strokes that does not overlap with the first set of strokes; The incremental trajectory is determined based on the structured stroke data corresponding to each independent stroke in the third stroke set.

10. The method according to claim 9, characterized in that, Before retrieving the full amount of structured stroke data from the first buffer, the method further includes: The sensor module collects multi-dimensional sensor data of the user's drawn trajectory; the multi-dimensional sensor data includes at least one of the following: screen touch position coordinate information; screen touch position pressure sensitivity information; touch movement speed information; and timestamp sequence information corresponding to the screen touch position coordinate information. Based on the multi-dimensional sensor data, at least one independent stroke is determined according to the user's input state when drawing a single stroke; wherein, the input state is used to characterize the state of the user's interactive action when drawing a single stroke. Based on each independent stroke in the at least one independent stroke and the corresponding multi-dimensional sensor data, the structured stroke data of each independent stroke is determined.

11. The method according to claim 10, characterized in that, The step of determining at least one independent stroke based on the multi-dimensional sensor data and the user's input state when drawing a single stroke includes: Obtain the first multi-dimensional sensor data within the sliding time window from the multi-dimensional sensor data; Based on the user's input state of drawing a single stroke, at least one independent stroke is generated based on the second multi-dimensional sensor data in the first multi-dimensional sensor data; The second multi-dimensional sensor data is the multi-dimensional sensor data in the first multi-dimensional sensor data that was not used to generate independent strokes.

12. The method according to claim 10, characterized in that, The structured stroke data of each independent stroke is stored in the first buffer; before obtaining the structured trajectory data of the first pattern from the feature cache pool, the method further includes: If the amount of structured stroke data stored in the first buffer is greater than or equal to the third threshold, the structured stroke data stored in the first buffer is copied to the second buffer, and the structured stroke data stored in the first buffer is deleted. From the second buffer, obtain the structured stroke data of each individual stroke to determine at least one trajectory drawn by the user; Based on the at least one trajectory, determine the geometric feature information corresponding to each trajectory in the at least one trajectory; The first pattern is determined based on the geometric feature information corresponding to each trajectory; The structured stroke data corresponding to the first pattern and the geometric feature information corresponding to the first pattern are determined as the structured trajectory data of the first pattern, and the structured trajectory data of the first pattern is stored in the feature cache pool.

13. The method according to claim 1, characterized in that, The method further includes: Receive a second input for the intended drawing pattern; In response to the second input, based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern, it is determined whether the first graphic drawn by the user is a complete graphic; If the first graphic is incomplete, the first graphic is completed based on the main graphic and auxiliary graphic corresponding to the drawing intention pattern to obtain the third graphic.

14. The method according to claim 1, characterized in that, The method further includes: Receive a third input for a stop drawing control or for the drawing intention pattern; In response to the third input, an image frame is generated for each stroke in the stroke set drawn by the user in the first graphic, based on the stroke set drawn by the user. A pattern drawing video is generated based on the image frames of each stroke and the image frames corresponding to the drawing intention pattern; the pattern drawing video includes image frames arranged in the order in which the user draws the strokes; the first or last image frame of the pattern drawing video is the image frame corresponding to the drawing intention pattern; Play the image to draw a video.

15. The method according to claim 1, characterized in that, The drawing intention pattern includes drawing intention patterns of a first type of painting style and drawing intention patterns of a second type of painting style; the method further includes: The drawing intent pattern of the first type of painting style is displayed according to the first display size, and the drawing intent pattern of the second type of painting style is displayed according to the second display size; Wherein, the first display size is larger than the second display size; and the confidence level of the first type of painting style is greater than the confidence level of the second type of painting style.

16. A pattern display device, characterized in that, include: A receiving module is used to receive a first input when the first pattern is displayed; The first input is the user's input for drawing the trajectory; A display module is used to display at least one drawing intent pattern; The at least one drawing intention pattern is determined based on a second pattern and reference information. The second pattern is composed of an incremental trajectory corresponding to the first input and the first pattern. The reference information includes at least one of the following: user chat history and note information. The incremental trajectory includes the user drawing trajectory.

17. An electronic device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the pattern display method as described in any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the pattern display method as described in any one of claims 1-15.

Images