Icon insertion method and device for collaborative drawing board, electronic equipment and storage medium

Abstract

The invention provides an icon insertion method and device for a collaborative drawing board, electronic equipment and a storage medium. The method comprises the following steps: acquiring an icon insertion request for inserting an icon in a target collaborative drawing board, and extracting an icon insertion prompt text from the icon insertion request; obtaining element attribute data of at least one associated drawing board element in the target collaborative drawing board based on the icon insertion request; determining a target icon in a preset candidate icon set according to the icon insertion prompt text and the element attribute data; and inserting the target icon into the target collaborative drawing board. According to the embodiment of the invention, the efficiency of inserting the icons in the collaborative drawing board can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for inserting icons into a collaborative drawing board. Background Technology

[0002] Collaborative whiteboards are tools that allow multiple people to collaborate in real time. They offer a wealth of drawing templates, diverse drawing tools, and real-time collaboration features to support efficient team drawing and creative generation. Online collaborative whiteboards provide a large number of icons for users to choose from while drawing.

[0003] When an object wants to draw using icons, it can select a target icon that can express the purpose of the drawing from a large number of candidate icons and then insert it into the collaboration artboard.

[0004] However, among related technologies, inserting icons into collaborative canvases is inefficient. Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, and storage medium for inserting icons into a collaborative canvas, which can improve the efficiency of inserting icons into a collaborative canvas.

[0006] According to one aspect of this disclosure, a method for inserting icons into a collaborative canvas is provided, comprising:

[0007] Obtain the icon insertion request for inserting an icon into the target collaboration artboard, and extract the icon insertion prompt text from the icon insertion request;

[0008] Based on the icon insertion request, obtain the element attribute data of at least one associated canvas element in the target collaborative canvas;

[0009] Based on the icon, insert the prompt text and the element attribute data to determine the target icon from the preset candidate icon set;

[0010] Insert the target icon into the target collaboration canvas.

[0011] According to one aspect of this disclosure, an icon insertion device for a collaborative drawing board is provided, comprising:

[0012] The first acquisition unit is used to acquire an icon insertion request for inserting an icon into the target collaborative canvas, and extract icon insertion prompt text from the icon insertion request.

[0013] The second acquisition unit is used to acquire element attribute data of at least one associated canvas element in the target collaborative canvas based on the icon insertion request;

[0014] The determining unit is used to determine the target icon from a preset candidate icon set based on the inserted prompt text of the icon and the element attribute data;

[0015] An insertion unit is used to insert the target icon into the target collaboration canvas.

[0016] Optionally, the second acquisition unit is specifically used for:

[0017] Obtain at least one candidate canvas element from the target collaborative canvas, and obtain the correlation score between each candidate canvas element and the icon insertion request;

[0018] Based on the correlation score corresponding to each candidate canvas element, at least one associated canvas element is determined from at least one candidate canvas element, and the element attribute data of at least one associated canvas element is obtained.

[0019] Optionally, the second acquisition unit is specifically used for:

[0020] Obtain the element position of each candidate artboard element in the target collaborative artboard;

[0021] Based on the icon insertion request, obtain the target insertion position for inserting the icon in the target collaboration canvas;

[0022] Based on the target insertion position and the element position, a correlation score is determined between each candidate canvas element and the icon insertion request.

[0023] Optionally, the second acquisition unit is specifically used for:

[0024] Based on the icon insertion request, obtain the target insertion position and the insertion object of the icon in the target collaboration canvas;

[0025] Determine the same object canvas element corresponding to the inserted object from at least one of the candidate canvas elements;

[0026] Based on the same canvas element, the target insertion position, and the element position, a correlation score is determined between each candidate canvas element and the icon insertion request.

[0027] Optionally, the determining unit is specifically used for:

[0028] The element attribute data is input into a preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set.

[0029] At least one second candidate icon is determined from the preset candidate icon set based on the insertion probability;

[0030] The target icon is determined from the at least one second candidate icon based on the inserted prompt text.

[0031] Optionally, the determining unit is specifically used for:

[0032] Feature extraction is performed on the inserted prompt text of the icon to obtain the prompt text features;

[0033] For each second candidate icon, feature extraction is performed to obtain the candidate icon features;

[0034] Based on the features of the prompt text and the features of the candidate icons corresponding to each second candidate icon, the target icon is determined from the at least one second candidate icon.

[0035] Optionally, the determining unit is specifically used for:

[0036] The icon insertion prompt text is input into a first preset sub-neural network, and feature extraction is performed on the icon insertion prompt text to obtain text semantic features;

[0037] The second preset sub-neural network is used to extract key features from the semantic features of the text to obtain the prompt text features.

[0038] Optionally, the determining unit is specifically used for:

[0039] Retrieve the historical icon records corresponding to the target collaboration canvas;

[0040] Based on the historical icon records, determine the preference coefficient corresponding to each of the first candidate icons;

[0041] Based on the preference coefficient and the insertion probability corresponding to each first candidate icon, at least one second candidate icon is determined from the preset candidate icon set.

[0042] Optionally, the insertion unit is specifically used for:

[0043] Obtain the canvas layout information and object operation information of the target collaborative canvas;

[0044] Based on the canvas layout information and the object operation information, the target insertion position is determined in the target collaborative canvas;

[0045] Insert the target icon into the target insertion position on the target collaboration canvas.

[0046] Optionally, the insertion unit is specifically used for:

[0047] Based on the element attribute data corresponding to each of the associated canvas elements, the degree of association between the target icon and each of the associated canvas elements is determined;

[0048] Based on the canvas layout information, the object operation information, and the degree of association corresponding to each associated canvas element, the target insertion position is determined in the target collaborative canvas.

[0049] Optionally, the preset candidate icon set is updated in the following way:

[0050] Get the preset update cycle;

[0051] Based on the preset update cycle, obtain the number of times each candidate icon in the preset candidate icon set is used, and determine the low-frequency icons in the preset candidate icon set based on the number of times each candidate icon is used.

[0052] Obtain newly added icons from the target icon library during the preset update cycle, and update the preset candidate icon set based on the newly added icons and the low-frequency icons.

[0053] In the icon insertion method of the collaborative canvas of this disclosure embodiment, an icon insertion request for inserting an icon into the target collaborative canvas is obtained, and an icon insertion prompt text is extracted from the icon insertion request; element attribute data of at least one associated canvas element in the target collaborative canvas is obtained based on the icon insertion request; a target icon is determined from a preset candidate icon set according to the icon insertion prompt text and the element attribute data; and the target icon is inserted into the target collaborative canvas.

[0054] Therefore, embodiments of this disclosure can determine the icon to be inserted into a target collaboration artboard based on associated artboard elements and icon insertion prompt text. Since the icon to be inserted into the target collaboration artboard may have a close relationship with existing associated artboard elements, the target icon can be predicted based on the element attribute data of the associated artboard elements. Furthermore, the icon insertion prompt text can guide the prediction of the target icon, thereby finding the target icon more quickly from a preset candidate icon set. Compared to manually searching for the target icon in a large preset candidate icon set, determining the target icon automatically based on the association between artboard elements and the icon insertion prompt text leverages the relationships between artboard elements and text guidance. This improves the efficiency of inserting icons into collaboration artboards.

[0055] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0056] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0057] Figure 1 This is a system architecture diagram of the system to which the icon insertion method for the collaborative whiteboard according to an embodiment of the present disclosure is applied;

[0058] Figure 2A This is a schematic diagram illustrating an embodiment of the present disclosure applied in a collaborative drawing scenario.

[0059] Figure 2B This is another schematic diagram illustrating the application of the embodiments of this disclosure in a collaborative drawing scenario;

[0060] Figure 3 This is a flowchart of an icon insertion method for a collaborative canvas according to an embodiment of the present disclosure;

[0061] Figure 4A This is a schematic diagram illustrating the determination of the position of canvas elements based on a Cartesian coordinate system corresponding to a target collaborative canvas according to an embodiment of the present disclosure;

[0062] Figure 4B This is another schematic diagram illustrating the determination of the position of canvas elements based on the Cartesian coordinate system corresponding to the target collaborative canvas according to an embodiment of the present disclosure;

[0063] Figure 5A This is a schematic diagram illustrating the determination of a target insertion position according to an embodiment of the present disclosure;

[0064] Figure 5B This is another schematic diagram illustrating the determination of a target insertion position according to an embodiment of the present disclosure;

[0065] Figure 6 This is a schematic diagram illustrating the determination of the relevance score of each candidate canvas element based on the element position and the target insertion position according to an embodiment of the present disclosure.

[0066] Figure 7 This is a schematic diagram of displaying multiple candidate icons on a target collaboration canvas according to an embodiment of the present disclosure;

[0067] Figure 8 This is a schematic diagram illustrating the acquisition of icons based on a search engine according to an embodiment of the present disclosure;

[0068] Figure 9 This is a schematic diagram illustrating the prediction of insertion probability based on multiple element attribute data according to an embodiment of the present disclosure;

[0069] Figure 10This is a schematic diagram illustrating the determination of a preset position based on the layout of associated canvas elements in a target collaborative canvas according to an embodiment of the present disclosure.

[0070] Figure 11 This is another flowchart illustrating a method for inserting icons into a collaborative canvas according to an embodiment of the present disclosure;

[0071] Figure 12 This is a schematic diagram of the core technical architecture according to an embodiment of the present disclosure;

[0072] Figure 13 This is a schematic diagram of the structure of an icon insertion device for a collaborative drawing board according to an embodiment of the present disclosure;

[0073] Figure 14 This is a terminal structure diagram for implementing various methods according to an embodiment of the present disclosure;

[0074] Figure 15 This is a server structure diagram illustrating the implementation of various methods according to an embodiment of the present disclosure. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0076] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0077] Collaborative canvases: A tool that allows multiple people to edit and share canvases online simultaneously, ideal for remote work and team collaboration.

[0078] Neural Networks: Neural networks are an important technique in machine learning, inspired by the structure and function of the human brain. A neural network consists of a large number of neurons that interact through different connections (weights), forming a complex network. Neural networks can automatically learn features from data and handle complex problems through multi-layered network structures.

[0079] In related technologies, when an object wants to insert an icon into a collaboration canvas, it often manually browses the icon set corresponding to the collaboration canvas and searches for the desired target icon. Since the icon set contains a large amount of data, to reduce browsing time, the object can enter keywords in the icon search box and manually select the target icon from the submitted candidate icons. While this reduces the number of candidate icons, it does not fundamentally solve the problem of low icon insertion efficiency caused by manual search. Therefore, this disclosure provides an icon insertion method for a collaboration canvas, aiming to improve the accuracy of icon insertion in collaboration canvases.

[0080] System architecture and scenario description of the embodiments disclosed herein

[0081] Figure 1 This is a system architecture diagram applied to the icon insertion method of the collaborative whiteboard according to an embodiment of the present disclosure. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0082] Terminal 140 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple devices can be connected via a local area network, sharing a single display device to work collaboratively, forming a single terminal 140. Terminal 140 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0083] Server 110 refers to a computer system that can provide certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a machine), or a combination of portions of multiple high-performance computers (e.g., machines).

[0084] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator between two systems using different communication protocols, data formats, languages, or even completely different architectures. It enables network interconnection at the transport layer. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 through gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 through gateway 120.

[0085] The icon insertion method for the collaborative whiteboard in this embodiment can be executed by an electronic device, which can be a terminal 140 or a server 110. That is, the icon insertion method for the collaborative whiteboard can be executed by the terminal 140 or the server 110, or it can be executed by both the terminal 140 and the server 110.

[0086] The embodiments disclosed herein can be applied in various scenarios, such as Figures 2A-2B The collaborative drawing scenario shown.

[0087] like Figure 2A As shown, the collaborative canvas 210 includes a drawing area 211 and an icon insertion control area 212. The icon insertion control area 212 includes a prompt text input box 213 and an icon insertion control 214. In the drawing area 211, icons corresponding to "Text Content T1", "Text Content T2", "Text Content T3", and "Text Content T3" are drawn sequentially, and the relationships between the various elements are shown through connecting lines.

[0088] After adding "Text Content T4" to the drawing area 211, an icon matching "Text Content T4" needs to be added. The various elements already displayed in the drawing area 211 (including text elements, icon elements, and connecting line elements) can all serve as associated canvas elements for predicting the target icon. Additionally, the desired target icon's hint text can be added to the hint text input box 213. After entering the hint text, the icon insertion control 214 can be triggered to determine the target icon to be added after "Text Content T4" based on the hint text and associated canvas elements.

[0089] like Figure 2B As shown, based on the prompt text and associated canvas elements, it can be determined that the target icon is a mobile phone-style icon. Therefore, the target icon can be inserted after "text content T4". In this way, the drawing object does not need to manually search for the target icon in the icon set, but can quickly obtain the target icon based on the associated canvas elements in drawing area 211 and the icon insertion prompt text, which greatly improves the efficiency of icon insertion and thus improves the drawing efficiency of collaborative drawing.

[0090] General Description of Embodiments in this Disclosure

[0091] According to one embodiment of this disclosure, a method for inserting icons into a collaborative drawing board is provided. The icon insertion method for a collaborative drawing board provided in this embodiment can be applied to, for example... Figures 2A-2B In scenarios such as collaborative drawing, as shown. Figure 3 The diagram shown is a flowchart illustrating a method for inserting icons into a collaborative canvas provided in this disclosure. This method may include:

[0092] Step 310: Obtain the icon insertion request for inserting an icon into the target collaboration artboard, and extract the icon insertion tooltip text from the icon insertion request.

[0093] The target collaboration artboard can be the collaboration artboard where an icon needs to be inserted. An icon insertion request expresses a request to insert an icon into the target collaboration artboard. An icon insertion request can be initiated by a drawing object using the target collaboration artboard. The target collaboration artboard can include an icon insertion control for initiating the icon insertion request, such as... Figure 2A The icon insertion control 214 is shown below. When a drawing object needs to use an icon while drawing on a target collaborative artboard, it can initiate an icon insertion request by triggering the icon insertion control.

[0094] The icon insertion request can include icon insertion tooltip text. This tooltip text guides and prompts the search for the target icon to be inserted in the collaboration artboard. For example, if the tooltip text is "Create a schedule," then the target icon could be an icon related to a schedule. The icon insertion tooltip text can be text content generated by the drawing object based on the drawing requirements. The target collaboration artboard can contain an icon insertion tooltip text input control, such as... Figure 2A The example shown is the prompt text input box 213. When an object needs to insert an icon, the prompt text can be entered in the icon insertion prompt text input control, which then triggers the icon insertion control to generate an icon insertion request containing the prompt text.

[0095] Step 320: Obtain the element attribute data of at least one associated artboard element in the target collaboration artboard based on the icon insertion request.

[0096] The associated artboard element can be any element in the target collaboration artboard that is associated with the target icon to be inserted. It can be any type of artboard element in the target collaboration artboard, such as text elements, table elements, icon elements, or graphic elements.

[0097] Element attribute data can be data corresponding to associated canvas elements that express the content of those elements. For different types of associated canvas elements, their corresponding element attribute data can be of different types. For example, the element attribute data for text-based associated canvas elements can be text content data; the element attribute data for table-based associated canvas elements can be table content data; the element attribute data for icon-based associated canvas elements can be text-based data corresponding to the meaning expressed by the icon; and the element attribute data for graphic-based associated canvas elements can be text-based data corresponding to the meaning expressed by the graphic.

[0098] Multiple artboard elements within a collaboration artboard may be related. For example, after adding text to a collaboration artboard, the icon you want to add around the text might indicate the meaning or keywords of that text. Similarly, when drawing a tree diagram in a collaboration artboard, the artboard elements corresponding to multiple child nodes under a node might describe the same type of content. For instance, if a node describes "electronic devices," its child nodes might have multiple artboard elements such as a "computer" icon, a "phone" icon, and a "printer" icon. Therefore, the target icon can be predicted based on the element attribute data of the related artboard elements in the collaboration artboard that are associated with the target icon to be inserted.

[0099] In one implementation, upon receiving an icon insertion request, all artboard elements contained in the target collaborative artboard can be obtained as associated artboard elements, and the element attribute data corresponding to each associated artboard element can be obtained.

[0100] In another implementation, based on the icon insertion request, the element attribute data of at least one associated artboard element in the target collaboration artboard is obtained, including:

[0101] Get at least one candidate artboard element from the target collaboration artboard, and get the relevance score between each candidate artboard element and the icon insertion request;

[0102] Based on the relevance score corresponding to each candidate canvas element, at least one associated canvas element is determined from at least one candidate canvas element, and the element attribute data of at least one associated canvas element is obtained.

[0103] To obtain at least one candidate artboard element from the target collaboration artboard, you can use all artboard elements contained in the target collaboration artboard as candidate artboard elements.

[0104] The correlation score between candidate canvas elements and icon insertion requests indicates the degree of correlation between candidate canvas elements and icon insertion requests, that is, the degree of correlation between candidate canvas elements and target icon predictions.

[0105] In one implementation, obtaining the relevance score between each candidate canvas element and the icon insertion request includes:

[0106] Get the position of each candidate artboard element in the target collaborative artboard;

[0107] Get the target insertion position of the icon in the target collaboration artboard based on the icon insertion request;

[0108] The relevance score between each candidate canvas element and the icon insertion request is determined based on the target insertion position and the element position.

[0109] The element position of the candidate artboard element indicates the position information of the candidate artboard element within the target collaboration artboard.

[0110] The positional information of canvas elements in the target collaboration canvas can be determined based on a Cartesian coordinate system created based on the target collaboration canvas. For example... Figure 4A and Figure 4B As shown, with the lower left corner of the target collaborative drawing board 410 as the origin, an x-axis is constructed horizontally to the right, and a y-axis is constructed vertically upwards perpendicular to the x-axis, thus obtaining the Cartesian coordinate system corresponding to the target collaborative drawing board 410.

[0111] In one implementation, for canvas elements in a collaborative canvas, the position coordinates of the canvas element's center can be obtained in a Cartesian coordinate system as the canvas element's position information. For example... Figure 4A In the target collaborative canvas 410, the center point of "text content T1" is located at coordinates (7.65, 19.25), which is used as the position coordinates of the canvas element "text content T1".

[0112] In another implementation, for canvas elements in a collaborative canvas, the vertex coordinates of the polygonal region where the canvas element is located can be obtained in a Cartesian coordinate system as the position information of the canvas element.

[0113] Because different artboard elements in a collaborative artboard have different sizes, when the artboard element size is large, using the center position of the artboard element as its position may not accurately describe the distribution of artboard elements within the collaborative artboard. Therefore, a polygonal region can be defined within the collaborative artboard to contain each artboard element, and the position information of the artboard element can be determined using the coordinates of each vertex of the polygonal region. For example... Figure 4B As shown, the polygonal area containing the canvas element "Text Content T1" is a rectangular area formed by four points A, B, C, and D. The coordinates of point A are (4.3, 20), the coordinates of point B are (4.3, 18.5), the coordinates of point C are (10, 18.5), and the coordinates of point D are (10, 20). Thus, the position information of the canvas element "Text Content T1" is: ((4.3, 20), (4.3, 18.5), (10, 18.5), (10, 20)).

[0114] The position of a candidate artboard element in the target collaboration artboard can be the position coordinates of the candidate artboard element in the Cartesian coordinate system corresponding to the target collaboration artboard, the coordinates of the center point of the candidate artboard element, or the coordinates of multiple vertices of the polygon in which the candidate artboard element is located.

[0115] After obtaining the element position of each candidate artboard element in the target collaboration artboard, the target insertion position for inserting the icon in the target collaboration artboard can be obtained based on the icon insertion request. The target insertion position can be the location where the icon is inserted in the target collaboration artboard.

[0116] In one implementation, the target insertion location can be specified by the drawing object. When a drawing object needs to insert an icon into a target collaboration artboard, the target insertion location can be specified by clicking on the desired location within the target collaboration artboard. For example... Figure 5A As shown, when a drawing object wants to insert an icon after "Text Content T4" and below the "Projector" icon in the target collaboration artboard 510, it can determine the target insertion position by clicking on that position. The coordinates of the target insertion position can be the coordinates of the drawing object's clicked position.

[0117] Determining the target insertion location can also be achieved by using the Region Limiting tool to define the area within the target collaboration artboard where the icon wants to be inserted. The Region Limiting tool can be used to define an area within the target collaboration artboard, thus restricting the drawing location. The Region Limiting tool can be a rectangle, a circle, or a custom shape. For example... Figure 5B As shown, the target collaboration artboard 510 includes a region limiting tool area 511, which includes a rectangle limiting tool, a circle limiting tool, and a custom shape limiting tool. When a drawing object wants to insert an icon after "Text Content T4" and below the "Projector" icon in the target collaboration artboard 510, the rectangle limiting tool is selected from the region limiting tool area 511, and a rectangle is drawn after "Text Content T4" and below the "Projector" icon by dragging or other means. This indicates that the position of the target icon to be inserted is limited within the rectangle, thus clarifying the target insertion position. The position coordinates corresponding to the target insertion position can be the position coordinates corresponding to the limiting area. In one embodiment, the position coordinates corresponding to the limiting area can be the center coordinates of the limiting area. In another embodiment, the position coordinates corresponding to the limiting area can include the coordinates of each vertex of the limiting area.

[0118] Once the target insertion position of the drawing object is determined in the target collaboration artboard, an icon insertion request can be generated. The icon insertion request can be generated based on the icon insertion prompt text described in step 310 and the target insertion position. Therefore, after obtaining the icon insertion request, the target insertion position for inserting the icon in the target collaboration artboard can be obtained based on the icon insertion request.

[0119] After determining the target insertion position and the element position corresponding to each candidate canvas element, the relevance score between each candidate canvas element and the icon insertion request can be determined based on the target insertion position and the element position.

[0120] In one implementation, determining the relevance score between each candidate canvas element and the icon insertion request based on the target insertion position and the element position includes:

[0121] Calculate the element distance based on the target insertion position and the element position;

[0122] The relevance score between each candidate canvas element and the icon insertion request is determined based on element distance.

[0123] Element distance represents the distance between the target insertion position and the element position. When both the target insertion position and the element position contain a center point coordinate, the element distance can be directly calculated based on the target insertion position and the element position. Methods for calculating the element distance based on the target insertion position and the element position include calculating the Euclidean distance between them. For example, if the coordinates of the target insertion position are (3, 6) and the coordinates of the element position are (1, 7), then the element distance between the target insertion position and the element position is:

[0124] When both the target insertion position and the element position contain coordinates corresponding to multiple vertices, the element distance between the target insertion position and the element position can be the distance between the two vertices closest to each other. The distance between vertices can also be calculated using Euclidean distance. For example, the target insertion position is ((2, 5), (2, 10), (5, 10), (5, 5)); the element position corresponding to the candidate canvas element is ((4, 7), (4, 3), (10, 3), (10, 7)). The distance between the coordinates of each vertex corresponding to the target insertion position and the coordinates of each vertex corresponding to the element position can be expressed as...

[0125] Table 1:

[0126] (2，5) (2，10) (5，10) (5，5) (4，7) 2.83 3.61 3.16 2.24 (4，3) 2.83 7.28 7.07 2.24 (10，3) 8.25 10.63 8.6 5.39 (10，7) 8.25 8.54 5.83 5.39

[0127] Table 1

[0128] As shown in Table 1, the closest vertices between the target insertion position and the corresponding element positions of the candidate canvas elements are the vertex (5, 5) in the target insertion position and the vertices (4, 7) and (4, 3) in the element positions, with a distance of 2.24. Therefore, 2.24 can be taken as the element distance between the target insertion position and the element positions.

[0129] After calculating the element distance between the element position of each candidate canvas element and the target insertion position, the relevance score between each candidate canvas element and the icon insertion request can be determined based on the element distance.

[0130] In one implementation, determining the correlation score based on element distance can be based on a preset lookup table, which indicates the correspondence between element distance and correlation score. The elements in the lookup table... The range of elemental distances can be based on The size is determined based on the target collaborative artboard. For example, as shown in Table 2:

[0131]

[0132]

[0133] Table 2

[0134] As shown in Table 2, when the element distance is 2.24, the correlation score between the candidate canvas element and the target insertion request is 90.

[0135] Determining the relevance score between candidate canvas elements and icon insertion requests based on a pre-defined comparison table can improve the efficiency of determining the relevance score.

[0136] In another implementation, determining the relevance score based on element distance can be achieved by sorting the element distances corresponding to at least one candidate canvas element and determining the relevance score based on the ascending order.

[0137] For example, candidate canvas elements include candidate canvas element W1, candidate canvas element W2, candidate canvas element W3, candidate canvas element W4, and candidate canvas element W5, with corresponding element distances of 3.74, 5.12, 1.65, 2.41, and 4.25, respectively. The result of sorting the candidate canvas elements based on element distance is: candidate canvas element W3, candidate canvas element W4, candidate canvas element W1, candidate canvas element W5, and candidate canvas element W2.

[0138] When determining relevance scores based on order, a relevance score can be assigned to each candidate canvas element according to a preset rule. The preset rule indicates the relevance score corresponding to different rankings. For example, the preset rule is: the highest-ranked candidate canvas element has a relevance score of 100, decreasing sequentially in increments of 10. When the relevance score corresponding to a candidate canvas element's ranking decreases to 0, the relevance scores for subsequent candidate canvas elements are all 0. For example, with 12 candidate canvas elements, after sorting them in ascending order of element distance, based on the above preset rule, the first-ranked candidate canvas element has a relevance score of 100, the second-ranked candidate canvas element has a relevance score of 90, the third-ranked candidate canvas element has a relevance score of 80, and so on. The ninth-ranked candidate canvas element has a relevance score of 10, and the candidate canvas elements ranked 10-12 have a relevance score of 0.

[0139] After sorting at least one candidate canvas element based on element distance, determining the correlation score based on the order relationship can be achieved more accurately. The closer the candidate canvas element is to the target insertion position, the higher its correlation score, which helps to improve the accuracy of determining the correlation score based on element distance.

[0140] Determining the relevance score between the candidate canvas element and the icon insertion request based on the element distance between the target insertion position and the corresponding element position of the candidate canvas element allows the length of the element distance to directly determine the relevance score, which helps to improve the efficiency of determining the relevance score between the candidate canvas element and the icon insertion request.

[0141] In another implementation, a correlation score between each candidate canvas element and the icon insertion request is determined based on the target insertion position and the element position, including:

[0142] Input the target insertion position and the element position corresponding to each candidate canvas element into a preset neural network model to obtain the correlation score between each candidate canvas element and the icon insertion request.

[0143] A pre-defined neural network model can be used to determine the relevance score of each candidate canvas element based on the element position corresponding to at least one candidate canvas element and the target insertion position. For example... Figure 6 As shown, the input features of the preset neural network model 603 include the target insertion position 601 and multiple element positions 602 corresponding to candidate canvas elements W1-Wn; the output of the preset neural network model 603 includes the correlation score 604 corresponding to each candidate canvas element.

[0144] The pre-defined neural network model can determine the overall element layout of at least one candidate canvas element in the target collaborative canvas based on the element position corresponding to each candidate canvas element, and determine the correlation score between each candidate canvas element and the target insertion request based on the overall element layout and the target insertion position. The model architecture of the pre-defined neural network model is not specifically limited and can be a convolutional neural network model or a recurrent neural network model, etc.

[0145] By using a pre-defined neural network model to determine the correlation score between each candidate canvas element and the icon insertion request, the correlation score of each candidate canvas element can be determined based on the overall element layout of at least one candidate canvas element, thus improving the accuracy of determining the correlation score.

[0146] In another implementation, obtaining the target insertion position for inserting an icon in the target collaboration artboard based on the icon insertion request includes: obtaining the target insertion position for inserting an icon in the target collaboration artboard and the insertion object based on the icon insertion request.

[0147] Therefore, based on the target insertion position and the element position, a correlation score is determined between each candidate canvas element and the icon insertion request, including:

[0148] Determine the corresponding canvas element of the inserted object from at least one candidate canvas element;

[0149] The relevance score between each candidate canvas element and the icon insertion request is determined based on the canvas element of the same object, the target insertion position, and the element position.

[0150] The method of obtaining the target insertion position for inserting an icon into the target collaboration artboard based on the icon insertion request has been described in detail in the preceding embodiments and will not be repeated here. The insertion object can be the object that initiates the icon insertion request and wants to insert an icon into the target collaboration artboard. The corresponding artboard element of the insertion object can be the artboard element drawn by the insertion object in the target collaboration artboard.

[0151] Collaborative artboards allow multiple objects to draw simultaneously on a single artboard. The degree of association between multiple artboard elements drawn by the same object may be higher than the degree of association between multiple artboard elements drawn by different objects. Therefore, based on the icon insertion request, the object to which the icon should be inserted into the target collaborative artboard can be determined, and then the artboard element drawn by the inserted object can be identified from at least one candidate artboard element.

[0152] If at least one candidate canvas element does not contain a canvas element of the same object, then the relevance score between each candidate canvas element and the target insertion request can be determined based on the target insertion position and the element position. This has been described in detail in the foregoing embodiments and will not be repeated here. If at least one candidate canvas element contains a canvas element of the same object, then the relevance score between each candidate canvas element and the target insertion request can be determined based on the canvas element of the same object, the target insertion position, and the element position.

[0153] In one implementation, determining the relevance score between each candidate canvas element and the icon insertion request based on the same canvas element, the target insertion position, and the element position includes:

[0154] Determine the positional correlation score between each candidate canvas element and the icon insertion request based on the target insertion position and the element position;

[0155] Get the first weight corresponding to the same canvas element, and the second weight corresponding to other candidate canvas elements other than the same canvas element;

[0156] The relevance score between each candidate canvas element and the icon insertion request is determined based on the first weight, the second weight, and the positional relevance score.

[0157] The process of determining the positional correlation score between each candidate canvas element and the icon insertion request based on the target insertion position and the element position is the same as the process of determining the correlation score between each candidate canvas element and the icon insertion request in the aforementioned implementation method, and will not be repeated here.

[0158] The first weight can be the weight corresponding to the same artboard element, and the second weight can be the weight corresponding to candidate artboard elements drawn by other objects, excluding the same artboard element. The first and second weights can be preset according to the needs of the actual application; for example, the first weight can be 1.2 and the second weight can be 1.

[0159] After obtaining the first and second weights, we can determine the relevance score based on the first and second weights and the positional relevance score corresponding to each candidate canvas element. Specifically, the positional relevance score corresponding to the canvas element with the same object is multiplied by the first weight to obtain the corresponding relevance score; the positional relevance scores corresponding to other candidate canvas elements besides the canvas element with the same object are multiplied by the second weight to obtain the corresponding relevance score. For example, the candidate canvas elements include: canvas element S1 with the same object, canvas element S2 with the same object, canvas element K1, canvas element K2, and canvas element K3, which are not the same object. Their corresponding positional relevance scores are 90, 75, 80, 78, and 60, respectively. When the first weight is 1.2 and the second weight is 1, their corresponding relevance scores are 108 (90*1.2), 90 (75*1.2), 80 (80*1), 78 (78*1), and 60 (60*1), respectively.

[0160] The correlation score between each candidate canvas element and the icon insertion request is determined based on the first weight, the second weight, and the position correlation score. The influence of canvas elements of the same object on the correlation score can be flexibly adjusted according to the needs of actual application, which improves the flexibility of determining the correlation score.

[0161] Determining the correlation score between each candidate canvas element and the icon insertion request based on the canvas elements of the same object, the target insertion position, and the element position can improve the influence of canvas elements drawn from the same object on the prediction of the target icon and improve the accuracy of determining the correlation score between candidate canvas elements and the icon insertion request.

[0162] Determining the relevance score between the candidate artboard element and the icon insertion request based on the target insertion position of the target icon and the element position of the candidate artboard element can improve the accuracy of determining the relevance score by determining the degree of relevance based on the positional relationship between artboard elements in the target collaborative artboard.

[0163] After obtaining the relevance score between each candidate canvas element and the icon insertion request, at least one associated canvas element can be determined from at least one candidate canvas element based on the relevance score corresponding to each candidate canvas element.

[0164] In one implementation, determining at least one associated canvas element from at least one candidate canvas element based on the relevance score corresponding to each candidate canvas element includes:

[0165] Sort at least one candidate canvas element in descending order of relevance score;

[0166] The candidate canvas elements that rank before the predetermined position in the sorting are identified as associated canvas elements.

[0167] For example, candidate canvas elements include: candidate canvas element W1, candidate canvas element W2, candidate canvas element W3, candidate canvas element W4, and candidate canvas element W5, with corresponding relevance scores of 70, 85, 65, 90, and 52, respectively. The result of sorting the candidate canvas elements according to their relevance scores is: candidate canvas element W4, candidate canvas element W2, candidate canvas element W1, candidate canvas element W3, and candidate canvas element W5. When the predetermined ranking is third, candidate canvas elements W4, W2, and W1, which are ranked higher than the predetermined ranking, can be identified as related canvas elements.

[0168] In another implementation, determining at least one associated canvas element from at least one candidate canvas element based on the relevance score corresponding to each candidate canvas element includes: obtaining a predetermined threshold and determining candidate canvas elements with a relevance score greater than the predetermined threshold as associated canvas elements.

[0169] For example, with a predetermined threshold of 80, based on the correlation scores of candidate canvas elements W1 to W5 in the above example, candidate canvas element W4 and candidate canvas element W2 can be identified as related canvas elements.

[0170] After identifying the associated artboard elements, you can obtain the element attribute data of the associated artboard elements.

[0171] Based on the relevance score between candidate canvas elements and icon insertion requests, at least one related canvas element is identified from the candidate canvas elements. Canvas elements with low relevance to the target icon prediction in the target collaboration canvas can be removed, while canvas elements with high relevance are retained. In this way, while reducing the amount of element attribute data and improving the efficiency of target icon prediction, it also avoids the influence of low-relevance canvas elements on target icon prediction, thus improving the accuracy of target icon prediction based on related canvas elements.

[0172] Step 330: Determine the target icon from the preset candidate icon set based on the icon insert prompt text and element attribute data.

[0173] The preset candidate icon set stores multiple candidate icons that can be inserted into the target collaboration artboard. These candidate icons can cover various themes, styles, and uses. For each candidate icon, the preset candidate icon set can also store corresponding descriptive information. The descriptive information can indicate the theme, style, color, shape, and other attributes of the corresponding candidate icon, as well as icon keywords and their meanings.

[0174] Multiple candidate icons from a preset set can be displayed on the target collaboration artboard, allowing drawing objects to visually view these candidate icons. For example... Figure 7 As shown, the target collaboration canvas 710 also includes an icon control area 720, which includes a prompt text input box 721 and a candidate icon display area 722. The prompt text input box 721 can be used to input prompt text for the icon insertion, and the candidate icon display area 722 can display multiple selectable candidate icons from a preset set of candidate icons.

[0175] In one implementation, the icon control area 720 can also be connected to an external search engine. When the candidate icon display area 722 does not contain the icon desired by the drawing object, a search command can be sent to the external search engine to retrieve the icon. The retrieved icon is then automatically added to a preset candidate icon set and displayed in the candidate icon display area 722. Figure 8 As shown, in the icon control area 720, when the desired icon is not found in the candidate icon display area 722, the address of an external search engine can be entered in the search engine input box 723 to find the desired icon. Furthermore, the icon control area 720 may also include a search engine display area 724, which displays multiple selectable search engines. The desired icon can be found by selecting a target search engine. Once the desired icon is found, it can be automatically added to a preset candidate icon set and displayed in the search engine display area 724.

[0176] In one implementation, the preset candidate icon set is updated in the following way:

[0177] Get the preset update cycle;

[0178] New icons are retrieved from the target icon library based on a preset update cycle, and the preset candidate icon set is updated based on the new icons.

[0179] To ensure that the candidate icons in the preset candidate icon set meet the needs of the drawing object, the preset candidate icon set can be updated according to a preset update cycle. The preset update cycle can be one week, one month, three months, etc.

[0180] Based on a preset update cycle, when the preset candidate icon set update time arrives, the system can retrieve newly added icons from the target icon library within a certain period. The target icon library can be an open-source, authoritative icon resource website. These websites possess abundant icon resources and regularly update and release new, high-quality icons.

[0181] When retrieving a new icon from a target icon library, you can either download the icon directly from the library or use web page parsing techniques to parse the icon content on the webpage, such as HTML parsers or regular expressions. HTML parsers can convert a webpage into a tree structure, allowing you to extract elements and attributes of icons at each level; regular expressions can help match icon formatting.

[0182] After obtaining the icon content, the icon data can be cleaned and formatted to obtain new icons. Data cleaning removes useless data and information from the obtained icon data, corrects errors, and fills in missing information. Formatting converts the icon data into a uniform format that can be stored within a preset set of candidate icons.

[0183] After obtaining a new icon, you can update the preset candidate icon set based on the new icon. Specifically, you can add the new icon to the preset candidate icon set.

[0184] Updating the preset candidate icon set based on a preset update cycle ensures that the candidate icons in the preset candidate icon set are updated regularly, thus improving the comprehensiveness of the candidate icons in the preset candidate icon set.

[0185] In another implementation, the preset candidate icon set is updated by: in response to the triggering of a preset event from the target icon library, obtaining new icons from the target icon library, and updating the preset candidate icon set based on the new icons.

[0186] Preset events can be pre-defined events used to trigger updates to a preset candidate icon set. For example, an icon has been updated in the target icon library. The target collaboration artboard can sense events occurring in the target icon library in real time. When a preset event is triggered, the newly added icon can be retrieved from the target icon library, and the preset candidate icon set can be updated based on the newly added icon. The process of retrieving the newly added icon and updating the preset candidate icon set based on the newly added icon will not be described in detail here.

[0187] Updating the preset candidate icon set based on the triggering of preset events can improve the real-time performance of the preset candidate icon set update.

[0188] In another implementation, the preset candidate icon set is updated in the following way:

[0189] Get the preset update cycle;

[0190] Based on a preset update cycle, obtain the number of times each candidate icon in the preset candidate icon set is used, and determine the low-frequency icons in the preset candidate icon set based on the number of times each candidate icon is used.

[0191] Retrieve newly added icons from the target icon library during a preset update cycle, and update the preset candidate icon set based on the newly added icons and low-frequency icons.

[0192] Based on a preset update cycle, when the preset candidate icon set update time is reached, the usage count of each candidate icon in the preset candidate icon set within the past cycle can be obtained. The icon usage count indicates how many times the candidate icon has been inserted into the collaboration artboard within the past cycle. When the usage count of a candidate icon is too low, it indicates that the icon may not conform to the usage habits of the drawing object, and therefore the candidate icon can be identified as a low-frequency icon.

[0193] When determining low-frequency icons based on their usage frequency, candidate icons whose usage frequency is below a preset threshold can be identified as low-frequency icons. For example, if the preset threshold is 10, then candidate icons whose usage frequency is below 10 are considered low-frequency icons.

[0194] The process of retrieving new icons from the target icon library has been described in detail in the preceding embodiments and will not be repeated here. When retrieving new icons and updating the preset candidate icon set with low-frequency icons, new icons can be added to the preset candidate icon set, and low-frequency icons can be removed from the preset candidate icon set.

[0195] Updating the preset candidate icon set based on newly added icons and infrequently used icons allows the preset candidate icon set to be updated promptly with newly added icons and remove infrequently used icons. This improves the real-time update speed of new icons while avoiding the problem of infrequently used icons occupying too much space and reducing the efficiency of icon insertion in the collaboration canvas.

[0196] Since the preset candidate icon set includes multiple candidate icons, the target icon can be determined from the preset candidate icon set based on the icon insert prompt text and element attribute data.

[0197] In one implementation, determining the target icon from a preset candidate icon set based on the icon insertion prompt text and element attribute data includes:

[0198] Input the element attribute data into a preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set;

[0199] At least one second candidate icon is determined from a preset set of candidate icons based on the insertion probability;

[0200] The target icon is determined from at least one second candidate icon based on the icon-insertion tooltip text.

[0201] The pre-defined neural network model can obtain the insertion probability of each first candidate icon in the pre-defined candidate icon set into the target collaborative canvas based on the element attribute data corresponding to at least one associated canvas element. Therefore, the pre-defined neural network model can be regarded as a multi-classification model, using the element attribute data corresponding to at least one associated canvas element as the model and multiple first candidate icons in the pre-defined candidate icon set as multiple categories, thereby predicting the insertion probability of each first candidate icon obtained based on the element attribute data corresponding to at least one associated canvas element.

[0202] Since element attribute data can be of different types—for example, text data corresponding to text-based canvas elements, icon data corresponding to image-based canvas elements, and table content data corresponding to table-based canvas elements—feature extraction can be performed on different types of element attribute data before inputting the element attribute data into the preset neural network model. Different feature extraction techniques can be used for different types of element attribute data. For example, word embedding can be used to extract text features for text-based element attribute data, while a convolutional neural network can be used to extract image features for image-based element attribute data. After extracting the element attribute features, the insertion probability of each first candidate icon can be predicted based on these features.

[0203] In one implementation, a pre-defined neural network model can treat the element attribute features corresponding to each associated canvas element as independent features, and then predict the insertion probability of each first candidate icon based on the element attribute features corresponding to at least one associated canvas element. The pre-defined neural network model can be a convolutional neural network model or a converter model, etc. Figure 9 As shown, multiple element attribute data 910 are subjected to feature extraction to obtain corresponding element attribute features 920. Then, multiple element attribute features are input into a preset neural network model 930 to obtain multiple insertion probabilities 940 corresponding to multiple first candidate icons.

[0204] In another implementation, element attribute data is input into a preset neural network model to obtain the insertion probability corresponding to each first candidate icon in the preset candidate icon set, including:

[0205] Construct an element attribute sequence based on the element attribute data corresponding to at least one associated canvas element;

[0206] Input the element attribute sequence into a preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set.

[0207] Since there are relationships between the elements in the target collaborative artboard, we can predict the insertion probability not only based on the element attribute features of each associated artboard element, but also by combining the relationships between the associated artboard elements.

[0208] When predicting insertion probability by combining the relationships between related canvas elements, an element attribute sequence can be constructed based on the element attribute data corresponding to at least one related canvas element. The element attribute sequence emphasizes the sequential order of the related canvas elements.

[0209] In one implementation, constructing an element attribute sequence based on element attribute data corresponding to at least one associated canvas element includes: constructing an element attribute sequence based on element attribute data in a preset order corresponding to at least one associated canvas element.

[0210] The preset order can be pre-defined based on the layout of related artboard elements in the target collaboration artboard. In one implementation, related artboard elements that appear earlier in the preset order have a greater impact on the prediction of the target icon. For example... Figure 10 As shown, in the target collaboration canvas 1010, the drawing object wants to add an icon after "Text Content T4" and below the "Projector" icon. The associated canvas elements include: "Text Content T4", "Text Content T1", the connecting line between "Text Content T4" and "Text Content T1", the "Projector" icon, and the "Printer" icon. The preset order of these associated canvas elements can be: "Text Content T4", the connecting line between "Text Content T4" and "Text Content T1", "Text Content T1", the "Projector" icon, and the "Printer" icon. Similarly, in another implementation, the earlier the associated canvas element appears in the color number order, the smaller its impact on the prediction of the target icon.

[0211] When constructing an element attribute sequence based on a preset order, if the preset order indicates that the influence of associated canvas elements on the prediction of the target icon is from low to high, the element attribute data corresponding to at least one associated canvas element can be sorted according to the preset order to obtain the element attribute sequence; if the preset order indicates that the influence of associated canvas elements on the prediction of the target icon is from high to low, the element attribute data corresponding to at least one associated canvas element can be sorted in reverse order of the preset order to obtain the element attribute sequence.

[0212] Predicting the target icon based on the element attribute sequence can be regarded as predicting the probability of the icon that may appear after the element attribute sequence. Therefore, sorting the element attribute sequence according to the order of the influence of the associated canvas elements on the target icon prediction from low to high is beneficial to improving the accuracy of the first candidate icon insertion probability prediction.

[0213] In one of the aforementioned embodiments, the associated artboard elements are determined based on the relevance score between candidate artboard elements in the target collaboration artboard and the icon insertion request. That is, different associated artboard elements may have different relevance scores with the target icon to be inserted. Based on this, in one embodiment, constructing an element attribute sequence based on element attribute data corresponding to at least one associated artboard element includes: constructing an element attribute sequence using element attribute data based on the relevance scores corresponding to at least one associated artboard element.

[0214] When constructing an element attribute sequence based on relevance scores, the sequence can be built in ascending order of relevance scores. For example, the associated canvas elements include: associated canvas element S1, associated canvas element S2, associated canvas element S3, and associated canvas element S4, with corresponding relevance scores of 87, 78, 92, and 80, respectively. Therefore, when constructing the element attribute sequence, the corresponding element attribute data can be sorted in the order of associated canvas element S3, associated canvas element S1, associated canvas element S4, and associated canvas element S2 to obtain the element attribute sequence. Thus, constructing the element attribute sequence in ascending order of relevance scores ensures that associated canvas elements with higher relevance to the target icon have a greater impact on the target icon prediction, thereby improving the accuracy of predicting the first candidate icon insertion probability based on the element attribute sequence.

[0215] Since element attribute data comes in different types, vectorization can be performed on different element attribute data to construct element attribute sequences. Vectorization can convert machine-incompatible text data, image data, or tabular data into machine-readable numerical forms.

[0216] After constructing the element attribute sequence, the element attribute sequence can be input into a preset neural network model to obtain the insertion probability corresponding to each first candidate icon in the preset candidate icon set.

[0217] The preset neural network model can be a recurrent neural network (RNN), or more specifically, a long short-term memory (LSTM) network. An RNN is a type of neural network that takes sequential data as input and uses a recurrent structure to process temporal dependencies within the sequence. A LSTM network is a special type of RNN that can solve the vanishing and exploding gradient problems encountered by traditional RNNs when processing long sequences of data. Therefore, based on the RNN model, the relationships between the element attribute data of each associated canvas element in the element attribute sequence can be captured, and the insertion probability of each first candidate icon can be predicted based on these relationships.

[0218] Predicting the insertion probability of each first candidate icon based on element attribute sequence can improve the accuracy of insertion probability prediction by predicting the insertion probability based on the sequence relationship between at least one associated canvas element.

[0219] After obtaining the insertion probability of each first candidate icon in the preset candidate icon set, at least one second candidate icon can be determined from the preset candidate icon set based on the insertion probability.

[0220] Since the preset candidate icon set may contain multiple icons with similar shapes or meanings, relying solely on the element attribute data of at least one associated canvas element may yield multiple icons with a high insertion probability. Therefore, the first candidate icon can be filtered based on its insertion probability to obtain at least one second candidate icon. The second candidate icon can be the first candidate icon with a high insertion probability from the preset candidate icon set.

[0221] In one implementation, determining at least one second candidate icon from a preset set of candidate icons based on insertion probability can be achieved by: sorting multiple first candidate icons in descending order of their corresponding insertion probabilities; and determining the first candidate icons preceding a predetermined order as second candidate icons. For example, the first candidate icons include: icon R1, icon R2, icon R3, icon R4, and icon R5, with corresponding insertion probabilities of 19%, 13%, 27%, 35%, and 6%, respectively. The sorting result according to insertion probability is: icon R4, icon R3, icon R1, icon R2, and icon R5. When the predetermined order is 3, the top 3 icons, namely icon R4, icon R3, and icon R1, can be determined as second candidate icons.

[0222] In another implementation, determining at least one second candidate icon from a preset set of candidate icons based on the insertion probability can be achieved by identifying first candidate icons with an insertion probability greater than a predetermined threshold as second candidate icons. For example, if the predetermined threshold is 25%, then based on the insertion probabilities corresponding to the multiple first candidate icons in the above example, the second candidate icons can be identified as icons R4 and R3.

[0223] In another implementation, determining at least one second candidate icon from a preset set of candidate icons based on the insertion probability includes:

[0224] Retrieve the historical icon records corresponding to the target collaboration artboard;

[0225] The preference coefficient for each first candidate icon is determined based on historical icon records.

[0226] Based on the preference coefficient and insertion probability corresponding to each first candidate icon, at least one second candidate icon is determined from the preset candidate icon set.

[0227] The icon history record for the target collaboration artboard can be a record of icons used on the target collaboration artboard within a predetermined time period between the current point in time. The icon history record can include the number of times each candidate icon in a preset set of candidate icons has been used. Therefore, the icon history record can reflect the preference for using various icons on the target collaboration artboard.

[0228] The preference coefficient reflects the degree of preference for the first candidate icon. The higher the preference coefficient, the more times the first candidate icon is used, and the higher the probability that it will be used again when predicting the target icon.

[0229] In one implementation, the preference coefficient corresponding to the first candidate icon can be determined based on a preset correlation table between usage frequency and preference coefficient. For example, Table 3 is an example of a correlation table between usage frequency and preference coefficient:

[0230] Number of times the first candidate icon is used Preference coefficient Greater than 20 1.6 16-20 1.4 11-15 1.2 5-10 1 Less than 5 0.8

[0231] Table 3

[0232] Based on Table 3, when the first candidate icon is used 18 times, its corresponding preference coefficient is 1.4.

[0233] After determining the preference coefficient corresponding to the first candidate icon, at least one second candidate icon can be determined from the preset candidate icon set based on the preference coefficient corresponding to each first candidate icon and the insertion probability. Specifically, the preference coefficient corresponding to each first candidate icon can be multiplied by the insertion probability to perform a preference shift on the insertion probability; at least one second candidate icon can be determined from the preset candidate icon set based on the shifted insertion probability.

[0234] For example, the first candidate icons include: icon R1, icon R2, icon R3, icon R4, and icon R5, with corresponding insertion probabilities of 19%, 13%, 27%, 35%, and 6%, and corresponding preference coefficients of 1, 1.2, 1.4, 1, and 1.2, respectively. The insertion probabilities after preference shifting for each first candidate icon are: 19% (19%*1), 15.6% (13%*1.2), 37.8% (27%*1.4), 35% (35%*1), and 7.2% (6%*1.2).

[0235] The method of determining at least one second candidate icon from the preset candidate icon set based on the insertion probability after offset is the same as the method of determining at least one second candidate icon from the preset candidate icon set based on the insertion probability in the aforementioned embodiments, and will not be repeated here.

[0236] The historical icon set can reflect the historical preference for inserting icons in the target collaboration artboard. Therefore, determining the second candidate icon from the preset candidate icon set based on the preference coefficient and insertion probability corresponding to each first candidate icon can take into account the icon usage preferences in the target collaboration artboard, which helps to improve the accuracy of determining the second candidate icon.

[0237] After identifying at least one second candidate icon, the target icon can be determined from the at least one second candidate icon based on the icon insert prompt text.

[0238] In one implementation, determining the target icon from at least one second candidate icon based on icon insertion prompt text includes:

[0239] Feature extraction is performed on the tooltip text inserted into the icon to obtain the tooltip text features;

[0240] For each second candidate icon, feature extraction is performed to obtain the candidate icon features;

[0241] The target icon is determined from at least one second candidate icon based on the features of the prompt text and the features of the candidate icon corresponding to each second candidate icon.

[0242] In one implementation, feature extraction is performed on the icon insertion prompt text, including:

[0243] The tooltip text inserted into the icon is cleaned and standardized to obtain preprocessed text;

[0244] The preprocessed text is vectorized to obtain text vectors;

[0245] The text vector is input into a preset neural network model for feature extraction to obtain the prompt text features.

[0246] Data cleaning of icon insertion tooltip text can include removing irrelevant characters, such as punctuation marks and special symbols, and removing erroneous text, such as spelling errors or typos.

[0247] The standardization process for the icon insertion prompt text after data cleaning can include: segmenting the icon insertion prompt text into words to obtain at least one word unit; and converting the at least one word unit into the same character encoding format to obtain preprocessed text.

[0248] Vectorization of preprocessed text is performed. Specifically, word embedding models can be used to convert preprocessed text data into numerical text vectors. Word embedding models map words to a high-dimensional space, making words with similar meanings closer together in the space; examples include Word2Vec and GloVe models. Using word embedding techniques to vectorize preprocessed text is beneficial for capturing the semantic relationships between words.

[0249] The text vectors are input into a pre-defined neural network model for feature extraction. This model can be a Long Short-Term Memory (LSTM) network or a Bidirectional Encoder Representations from Transformers (BERT) model. An LSTM model is a recurrent neural network that captures long-range dependencies in a text sequence, thereby extracting semantic context. A BERT model is a transformer-based model that learns bidirectional contextual relationships in a text sequence, enhancing semantic understanding. The features extracted from the text vectors by the pre-defined neural network model can indicate the sentiment and semantic relationships expressed in the icon insertion prompt text.

[0250] In another implementation, feature extraction is performed on the icon insertion prompt text to obtain prompt text features, including:

[0251] The icon insertion prompt text is input into the first preset sub-neural network, and the feature of the icon insertion prompt text is extracted to obtain the text semantic features;

[0252] The second preset sub-neural network is used to extract key features of the text semantic features to obtain the prompt text features.

[0253] The process of extracting features from the icon insertion prompt text based on the first preset sub-neural network model is the same as the process described in the previous embodiment, which involves extracting features from the icon insertion prompt text based on a preset neural network model to obtain the prompt text features. Therefore, it will not be repeated here. The first preset sub-neural network model can be an LSTM model or a BERT model.

[0254] When the icon insertion prompt text contains a lot of information, semantic feature extraction from only the prompt text may yield features corresponding to multiple key pieces of information. This can lead to key information confusion when matching the prompt text to the target icon, meaning it may be impossible to accurately identify which feature corresponds to the core feature of the target icon. For example, semantic feature extraction from the prompt text yields semantic features corresponding to key information K1, K2, and K3, where key information K3 is the core information used to determine the target icon. In this case, key information K1 and K2 can hinder the identification of the target icon. Therefore, after obtaining the text semantic features, a second pre-defined sub-neural network model can be used to extract associated features from the text semantic features, thereby obtaining the prompt text features. These prompt text features indicate the core information in the icon insertion prompt text used to insert the target icon.

[0255] The second pre-defined sub-neural network can extract key features from the semantic features of the text by extracting contextual keywords, analyzing the dependencies between text features, and analyzing the text's grammatical structure, thus obtaining the prompt text features. The second pre-defined sub-neural network can be a convolutional neural network or a converter model, etc.

[0256] Based on the first preset sub-neural network, feature extraction is performed on the icon insertion prompt text. Then, the second preset sub-neural network is used to extract key features from the semantic features of the text to obtain the prompt text features. This can avoid confusion of key information caused by too many elements in the icon insertion prompt text, thereby improving the accuracy of prompt text feature determination.

[0257] After extracting the features of the prompt text corresponding to the icon insertion prompt text, feature extraction can be performed on each second candidate icon to obtain the candidate icon features.

[0258] Feature extraction of the second candidate icon yields its corresponding descriptive information. Feature extraction is then performed on this descriptive information to obtain the candidate icon's features. The descriptive information for the second candidate icon has been described in detail above and may include attributes such as the icon's theme, style, color, and shape, as well as the icon's keywords and their meanings. Therefore, the process of feature extraction for the second candidate icon can also be considered a text feature extraction process. Consequently, the process of feature extraction for the second candidate icon can be the same as the process of feature extraction for the inserted tooltip text in the previous steps, and will not be repeated here.

[0259] After obtaining the prompt text features and the candidate icon features corresponding to each second candidate icon, the target icon can be determined from the second candidate icons based on the prompt text features and the candidate icon features.

[0260] In one implementation, determining a target icon from at least one second candidate icon based on the prompt text features and the candidate icon features corresponding to each second candidate icon includes: calculating a similarity score between the prompt text features and the candidate icon features corresponding to each second candidate icon, and determining the target icon from the second candidate icons based on the similarity score.

[0261] The similarity score between the prompt text features and the candidate icon features corresponding to the second candidate icon can be calculated using methods such as cosine similarity, Euclidean distance, or Mahalanobis distance.

[0262] After calculating the similarity score, the second candidate icons can be sorted based on the similarity score, and the icon with the highest similarity score can be selected as the target icon.

[0263] In another implementation, before determining the target icon from at least one second candidate icon based on the prompt text features and the candidate icon features corresponding to each second candidate icon, the method further includes:

[0264] Intent classification is performed based on icon-inserted prompt text to determine the target intent;

[0265] Thus, based on the features of the prompt text and the features of the candidate icons corresponding to each second candidate icon, the target icon is determined from at least one second candidate icon, including:

[0266] The target icon is determined from at least one second candidate icon based on the target intent, the features of the prompt text, and the features of the candidate icon corresponding to each second candidate icon.

[0267] The target intent, derived from the target insertion prompt text, can instruct the drawing object on how it wants to insert an icon. For example, the drawing object might want the target collaboration artboard to automatically insert a target icon; or it might want the target collaboration artboard to display multiple target icons, from which the drawing object can choose to insert one. In this way, the target collaboration artboard can perform corresponding operations based on the drawing object's target intent, thus fulfilling the drawing object's requirements.

[0268] The target intent of a drawing object can be expressed through target insertion prompt text. For example, if the target insertion prompt text is "Insert XXX icon," the target intent of the drawing object is to directly insert the target icon into the artboard. Another example is "Find icons related to XXX," which indicates that the target intent of the drawing object is to display multiple target icons for the drawing object to select from. Therefore, intents can be categorized and target intents determined based on icon insertion prompt text.

[0269] Intent classification can employ a pre-defined classifier. This classifier can be a machine learning model, such as a support vector machine or random forest; or a deep learning model, such as a convolutional neural network or a transformer model. The pre-defined classifier can be trained on sample text data pre-labeled with intent categories, thus enabling intent classification based on the inserted prompt text and the icon, thereby obtaining the target intent.

[0270] After determining the target intent, the target icon can be determined from the second candidate icons based on the target intent, the features of the prompt text, and the features of the candidate icons corresponding to each second candidate icon.

[0271] When the target intent is to directly insert an icon, a target icon can be determined from the second candidate icons based on the prompt text features and the candidate icon features corresponding to each second candidate icon. When the target intent is to display multiple target icons for the drawing object to select, at least one target icon can be determined from the second candidate icons based on the prompt text features and the candidate icon features corresponding to each second candidate icon, so that it can be displayed in the target collaboration artboard for the drawing object to choose from.

[0272] When determining a target icon based on the intention of directly inserting an icon, the second candidate icon with the highest similarity score can be selected as the target icon. When determining at least one target icon based on the intention of displaying multiple target icons for the drawing object to choose from, the target icons can be determined according to the similarity score ranking, with those having higher similarity scores being given priority as target icons to recommend to the drawing object.

[0273] Determining the target intent of a drawing object based on the inserted prompt text and then determining the target icon accordingly can make the recommended icon style more in line with the requirements of the drawing object, thereby improving the user experience of the target object when using the target collaboration artboard.

[0274] By comparing the features of the prompt text corresponding to the icon insertion prompt text with the features of the candidate icon corresponding to each second candidate icon, the target icon can be determined, ensuring the degree of matching between the target icon and the icon insertion prompt text, thus improving the accuracy of determining the target icon.

[0275] Based on the element attribute features corresponding to the associated canvas elements, the first candidate icons in the preset candidate icon set are filtered to obtain the second candidate icons; then, the target icon is determined from the second candidate icons based on the icon insertion prompt text. In this way, a preset neural network model performs one round of filtering on the preset candidate icon set, quickly obtaining at least one second candidate icon that can be inserted, and then accurately retrieving the target icon from the second candidate icons using the icon insertion prompt text. This improves the efficiency of target icon retrieval while ensuring that the target icon meets the requirements of the associated canvas elements and the target insertion prompt text, thus improving the accuracy of target icon retrieval.

[0276] Step 340: Insert the target icon into the target collaboration canvas.

[0277] Once the target icon is determined, it can be automatically inserted into the target collaboration canvas.

[0278] In one implementation, inserting the target icon into the target collaboration canvas includes:

[0279] Based on the icon insertion request, obtain the target insertion position of the icon in the target collaboration artboard, and insert the target icon into the target insertion position.

[0280] The process of obtaining the target insertion position based on the target insertion request has been described in detail above and will not be repeated here. After determining the target insertion position, the target icon can be inserted into the target insertion position. In the visualization interface of the drawing object, it can be clearly seen that the target icon has been inserted into the target insertion position of the target collaboration artboard.

[0281] In another implementation, inserting the target icon into the target collaboration canvas includes:

[0282] Obtain the canvas layout information and object operation information of the target collaborative canvas;

[0283] Determine the target insertion position in the target collaborative artboard based on artboard layout information and object operation information;

[0284] Insert the target icon into the target insertion position on the target collaboration canvas.

[0285] Object operation information can instruct the drawing object on the specified insertion position within the target collaboration artboard. For example, the desired insertion position can be specified by clicking on the desired location, or the area limitation tool can be used to define the area within the target collaboration artboard where the icon is to be inserted. Details have been described in detail in the foregoing embodiments and will not be repeated here. The specified icon insertion position can be determined using object operation information.

[0286] However, the icon insertion position specified by the object manipulation information may be rather coarse. When the target icon is inserted based on the object manipulation information, issues such as overlap with other artboard elements or poor layout may occur. Therefore, it is possible to obtain the artboard layout information of the target collaboration artboard and determine the target insertion position within the target collaboration artboard based on the artboard layout information and the object manipulation information.

[0287] The canvas layout information of the target collaboration canvas can include the position information corresponding to each canvas element in the target collaboration canvas. When determining the target insertion position based on the canvas layout information and object operation information, the canvas layout information and object operation information can be input into a preset neural network model. The preset neural network model can calculate the target insertion position based on the position information of each canvas element in the canvas layout information and the icon insertion position indicated by the object operation information.

[0288] In another implementation, determining the target insertion position in the target collaborative artboard based on artboard layout information and object operation information includes:

[0289] Based on the element attribute data corresponding to each associated canvas element, determine the degree of association between the target icon and each associated canvas element;

[0290] Based on the canvas layout information, object operation information, and the degree of association between each associated canvas element, the target insertion position is determined in the target collaborative canvas.

[0291] In the target collaborative artboard, two artboard elements with a high degree of correlation can be placed close together to allow the drawing object to obtain more drawing information. Therefore, the correlation between the target icon and each associated artboard element can be determined. The process of determining the correlation is the same as the process of determining the correlation score between each candidate artboard element and the icon insertion request in the aforementioned implementation method, and will not be repeated here.

[0292] After determining the degree of association, the target insertion position can be determined in the target collaborative artboard based on the artboard layout information, object operation information, and the degree of association corresponding to each associated artboard element. Specifically, the artboard layout information, object operation information, and the degree of association corresponding to each associated artboard element can be input into a preset neural network model. The preset neural network model can calculate the target insertion position based on the position information of each artboard element in the artboard layout information, the icon insertion position indicated by the object operation information, and the degree of association between the associated artboard elements and the target icon.

[0293] Determining the target insertion position based on canvas layout information, object operation information, and the degree of association of each associated canvas element can take into account the association between canvas elements, making the positions of canvas elements with a high degree of association closer together, which is conducive to improving the rationality of the layout of canvas elements in the target collaborative canvas.

[0294] After determining the target insertion location, the target icon can be inserted into the target insertion location on the target collaboration artboard.

[0295] The target insertion position is determined based on the canvas layout information and object operation information of the target collaborative canvas. This allows for adjustment of the target icon's insertion position based on the canvas layout information, which is based on the specified position of the object, thus improving the accuracy of the target insertion position determination.

[0296] In one implementation, inserting a target icon into a target collaboration artboard includes: adjusting the format and size of the target icon based on the format and size of the artboard elements in the target collaboration artboard; and inserting the adjusted target icon into the target collaboration artboard.

[0297] When inserting a target icon, you can ensure that its format and size are consistent with the format and size of the artboard elements in the target collaboration artboard. For example, you can convert the icon to SVG format or adjust the target icon's pixels according to the resolution of the target collaboration artboard.

[0298] In one implementation, when inserting a target icon into a target collaboration canvas within the visualization interface of the drawing object, an animation transition effect can be added to make the insertion process more natural and smooth. The animation transition effect can be implemented using CSS3 animations, JavaScript animation libraries (such as jQuery, GSAP, etc.), or HTML5 Canvas technology.

[0299] In one implementation, a real-time preview window can be provided in the visualization interface of the drawing object, so that the drawing object can view the position, size, style and other information of the target icon inserted into the target collaboration artboard through the real-time preview window, so that the drawing object can adjust, cancel or redo the insertion of the target icon in a timely manner.

[0300] In one implementation, during the automated insertion of a target icon into a collaborative artboard, fault information, such as network interruptions, access restrictions, and format incompatibility issues, can be detected in real time. Different fault messages can be automatically processed based on preset rules. Simultaneously, fault information can be promptly fed back to the drawing object, along with corresponding solutions. This ensures the stability and reliability of the automated insertion of the target icon.

[0301] In summary, the embodiments of this disclosure can determine the icon to be inserted into a target collaboration artboard based on associated artboard elements and icon insertion prompt text. Since the icon to be inserted into the target collaboration artboard may have a close relationship with existing associated artboard elements, the target icon can be predicted based on the element attribute data of the associated artboard elements. Furthermore, the icon insertion prompt text can guide the prediction of the target icon, thereby finding the target icon more quickly from a preset candidate icon set. Compared to manually searching for the target icon in a large preset candidate icon set, determining the target icon quickly by leveraging the relationships between artboard elements and text guidance based on associated artboard elements can improve the efficiency of inserting icons into collaboration artboards.

[0302] This disclosure provides a detailed description of embodiments in conjunction with specific application scenarios.

[0303] like Figure 11 The diagram shown illustrates the specific process of applying the collaborative canvas icon insertion method provided in this disclosure to a script summary generation scenario. The collaborative canvas icon insertion method includes:

[0304] Step 1101: Based on the preset update cycle, obtain the newly added icons in the target icon library, and update the preset candidate icon set based on the newly added icons.

[0305] Updating the preset candidate icon set according to a preset update cycle ensures that the preset candidate icon set includes the latest icon data. The preset update cycle can be one week, one month, three months, etc.

[0306] The target icon library can be an open-source, authoritative icon resource website. These websites have abundant icon resources and regularly update and release new high-quality icons. When the preset candidate icon set update time arrives, newly added icons from the target icon library over a period of time can be retrieved. When retrieving new icons from the target library, web page parsing techniques can be used to parse the icon content on the webpage, such as HTML parsers or regular expressions. After obtaining the icon content, the icon data can be cleaned and formatted to obtain the new icons.

[0307] After obtaining the new icon, you can add it to the preset candidate icon set.

[0308] Step 1102: Obtain the icon insertion request for inserting an icon into the target collaboration artboard.

[0309] An icon insertion request expresses a need to insert an icon into the target collaboration artboard. An icon insertion request can be initiated by a drawing object using the target collaboration artboard. When a drawing object needs to use an icon while drawing on the target collaboration artboard, it can initiate an icon insertion request by triggering the icon insertion control.

[0310] Step 1103: Extract the icon insertion prompt text from the icon insertion request, insert the target icon at the insertion position in the target collaboration artboard, and the drawing object.

[0311] The icon insertion request can include icon insertion tooltip text. This tooltip text guides and prompts users in finding the target icon to be inserted in the collaboration artboard. When an object needs to have an icon inserted, the user can enter the tooltip text in the tooltip text input control, which then triggers the icon insertion control to generate an icon insertion request containing the tooltip text.

[0312] The insertion location for a target icon in the target collaboration artboard can be specified by the drawing object. When a drawing object needs to insert an icon into the target collaboration artboard, it can specify the insertion location by clicking on the desired location within the target collaboration artboard. Once the drawing object has determined the insertion location in the target collaboration artboard, it can generate an icon insertion request.

[0313] Therefore, an icon insertion request can include icon insertion tooltip text, the insertion position of the target icon, and the drawing object that initiated the icon insertion request. Upon receiving an icon insertion request, it can be parsed to retrieve the icon insertion tooltip text, the insertion position of the target icon in the target collaboration artboard, and the drawing object.

[0314] Step 1104: Obtain at least one candidate artboard element in the target collaboration artboard, and the element position of each candidate artboard element in the target collaboration artboard.

[0315] Candidate artboard elements can be any artboard element contained in the target collaboration artboard. The element position of a candidate artboard element indicates its location within the target collaboration artboard.

[0316] The positional information of canvas elements in the target collaboration canvas can be determined based on a Cartesian coordinate system created based on the target collaboration canvas. The position of a candidate canvas element in the target collaboration canvas can be the position coordinates of the candidate canvas element in the corresponding Cartesian coordinate system of the target collaboration canvas.

[0317] Step 1105: Determine the correlation score between each candidate canvas element and the icon insertion request based on the drawing object, the insertion position, and the element position corresponding to each candidate canvas element.

[0318] When determining the relevance score between each candidate canvas element and the icon insertion request, the positional relevance score between each candidate canvas element and the icon insertion request can be determined based on the insertion position and the element position corresponding to each candidate canvas element.

[0319] The positional relevance score can be determined based on the element distance between the insertion position and the element position. The element distance can be obtained by calculating the Euclidean distance between the insertion position and the element position. After determining the element distance, multiple candidate canvas elements can be sorted according to their element distance, and a positional relevance score can be assigned to each candidate canvas element based on the sorting result and a preset rule. For example, the preset rule is: the positional relevance score corresponding to the highest-ranked candidate canvas element is 100, and it decreases sequentially in increments of 10. When the positional relevance score corresponding to the ranking of a candidate canvas element decreases to 0, the positional relevance score corresponding to subsequent candidate canvas elements is also 0.

[0320] After determining the positional relevance score, the corresponding artboard element for the drawing object is identified from the candidate artboard elements. Collaborative artboards support simultaneous drawing of multiple objects on a single artboard. A corresponding artboard element is an artboard element drawn by the drawing object and contained within the target collaborative artboard. Artboard elements in the target collaborative artboard that are not corresponding artboard elements are identified as non-corresponding artboard elements.

[0321] The positional relevance score is adjusted based on the first weight corresponding to the same canvas element and the second weight corresponding to the different canvas elements, and the relevance score between each candidate canvas element and the icon insertion request is determined.

[0322] The first weight can be the weight corresponding to the same artboard element, and the second weight can be the weight corresponding to candidate artboard elements drawn by other objects, excluding the same artboard element. The first and second weights can be preset according to the needs of the actual application; for example, the first weight can be 1.2, and the second weight can be 1. The positional relevance score corresponding to the same artboard element is multiplied by the first weight to obtain the corresponding relevance score; the positional relevance score corresponding to a different artboard element is multiplied by the second weight to obtain the corresponding relevance score.

[0323] Step 1106: Based on the correlation score corresponding to each candidate canvas element, determine at least one associated canvas element from at least one candidate canvas element, and obtain the element attribute data of at least one associated canvas element.

[0324] When determining at least one associated canvas element from at least one candidate canvas element based on the association score, a predetermined threshold can be obtained, and candidate canvas elements with an association score greater than the predetermined threshold can be determined as associated canvas elements. For example, if the predetermined threshold is 80, then candidate canvas elements with an association score greater than 80 can be determined as associated canvas elements.

[0325] After identifying the associated artboard elements, you can retrieve their element attribute data. This element attribute data can be data corresponding to the associated artboard elements and expressing their content. For different types of associated artboard elements, their corresponding element attribute data can be of different types.

[0326] Step 1107: Input the element attribute data into the preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set.

[0327] The preset neural network model can obtain the insertion probability of each first candidate icon in the preset candidate icon set into the target collaborative artboard based on the element attribute data corresponding to at least one associated artboard element.

[0328] Therefore, the preset neural network model can be regarded as a multi-classification model, which uses the element attribute data corresponding to at least one associated canvas element as the model and multiple first candidate icons in the preset candidate icon set as multiple categories, thereby predicting the insertion probability of each first candidate icon obtained based on the element attribute data corresponding to at least one associated canvas element.

[0329] Step 1108: Determine at least one second candidate icon from the preset candidate icon set based on the insertion probability.

[0330] Determining a second candidate icon from a preset set of candidate icons based on insertion probability can be achieved by: sorting multiple first candidate icons in descending order of their corresponding insertion probabilities; and selecting the first candidate icon preceding the predetermined order as the second candidate icon.

[0331] The second candidate icon can also be determined from the preset candidate icon set based on the insertion probability by: determining the first candidate icon with an insertion probability greater than a predetermined threshold as the second candidate icon.

[0332] Step 1109: Extract features from the icon insertion prompt text to obtain prompt text features, and extract features from each second candidate icon to obtain candidate icon features.

[0333] Feature extraction from the text inserted into the icon can begin by cleaning and standardizing the text to obtain preprocessed text. This preprocessed text is then vectorized to obtain text vectors. These text vectors are then input into a first pre-defined sub-neural network for feature extraction, yielding semantic features. This first pre-defined sub-neural network can be a Long Short-Term Memory (LSTM) network model or a bidirectional encoder representation model. The semantic features are then input into a second pre-defined sub-neural network for key feature extraction, resulting in the prompt text features. This second pre-defined sub-neural network can extract key features from the semantic features by extracting contextual keywords, analyzing dependencies between text features, and examining the text's grammatical structure. This second pre-defined sub-neural network can be a convolutional neural network or a converter model, among others.

[0334] Feature extraction for each second candidate icon yields its descriptive information. Textual feature extraction is then performed on this descriptive information to obtain the candidate icon's features. The second candidate icon can include attributes such as its theme, style, color, and shape, as well as its keywords and their meaning. The method for extracting features from the descriptive features of the second candidate icon is the same as that for extracting features from the icon's tooltip text, and will not be repeated here.

[0335] Step 1110: Based on the prompt text features and the candidate icon features corresponding to each second candidate icon, determine the target icon from at least one second candidate icon.

[0336] The target icon is determined from at least one second candidate icon by calculating the feature similarity between the feature of the prompt text and the feature of the candidate icon corresponding to each second candidate icon.

[0337] Feature similarity can be calculated using methods such as cosine similarity, Euclidean distance, or Mahalanobis distance.

[0338] After obtaining the feature similarity corresponding to each second candidate icon, the second candidate icons can be sorted in descending order of feature similarity, and the second candidate icons with high feature similarity can be recommended as target icons to the drawing object.

[0339] Step 1111: Obtain the artboard layout information and object operation information of the target collaborative artboard, and determine the target insertion position in the target collaborative artboard based on the artboard layout information and object operation information.

[0340] Object manipulation information can instruct drawing objects on actions to specify their insertion location within the target collaboration artboard. For example, you can specify the desired icon insertion location by clicking where you want to insert it, or you can use the region limiting tool to define the area within the target collaboration artboard where you want to insert the icon. Object manipulation information allows you to determine the specified icon insertion location for an object.

[0341] The layout information of the target collaboration artboard can include the position information of each artboard element in the target collaboration artboard.

[0342] When determining the target insertion position in the target collaborative canvas based on canvas layout information and object operation information, the canvas layout information and object operation information can be input into a preset neural network model. The preset neural network model can calculate the target insertion position based on the position information of each canvas element in the canvas layout information and the icon insertion position indicated by the object operation information.

[0343] Step 1112: Insert the target icon into the target insertion position on the target collaboration canvas.

[0344] Before inserting the target icon into the target collaboration artboard, you can adjust the format and size of the target icon based on the format and size of the artboard elements in the target collaboration artboard to ensure that the format and size of the target icon are consistent with the format and size of the artboard elements in the target collaboration artboard. Then, insert the target icon into the target collaboration artboard.

[0345] In the visualization interface of the drawing object, when inserting the target icon into the target collaboration artboard, an animation transition effect can be added to make the insertion process more natural and smooth. The drawing object can view the position, size, style, and other information of the target icon inserted into the target collaboration artboard through a real-time preview window, so that the drawing object can adjust, undo, or redo the insertion of the target icon in a timely manner.

[0346] During the automated insertion of target icons into the target collaboration artboard, fault information can be detected in real time, such as network interruptions, permission restrictions, and format incompatibility issues. Different fault messages can be automatically processed based on preset rules. Simultaneously, fault information can be promptly fed back to the drawing object, along with corresponding solutions. This ensures the stability and reliability of the automated insertion of target icons.

[0347] In summary, the core technical architecture corresponding to the icon insertion method for the collaborative whiteboard in this embodiment of the present disclosure is as follows: Figure 12 As shown, the process includes: constructing a preset candidate icon set; determining the target icon; and icon insertion. Constructing the preset candidate icon set includes: determining the target icon library; webpage parsing; data cleaning and formatting; and updating the icons. Determining the target icon includes: calculating the insertion probability of candidate icons; semantic analysis of the icon insertion prompt text; candidate icon feature extraction; similarity calculation; and candidate icon ranking and recommendation. The semantic analysis of the icon insertion prompt text includes: data preprocessing; generating text vectors; generating semantic features; and extracting key features. Data preprocessing includes word segmentation, data cleaning, and standardization; word embedding is used to generate the text vectors. Icon insertion includes: automatically determining the target insertion position; icon format adjustment; animation transitions; real-time preview; and error feedback.

[0348] Description of apparatus and devices according to embodiments of this disclosure

[0349] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0350] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target content, such as target content attribute information or attribute information sets, permission or consent from the target content provider will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining target content attribute information, separate permission or consent from the target content provider will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the separate permission or consent from the target content provider will the necessary target content-related data for the normal operation of the embodiments of this application be obtained.

[0351] Figure 13 A schematic diagram of the icon insertion device 1300 for a collaborative drawing board provided in this embodiment of the present disclosure. The icon insertion device 1300 for the collaborative drawing board includes:

[0352] The first acquisition unit 1310 is used to acquire an icon insertion request for inserting an icon into the target collaborative canvas, and extract icon insertion prompt text from the icon insertion request.

[0353] The second acquisition unit 1320 is used to acquire element attribute data of at least one associated canvas element in the target collaborative canvas based on the icon insertion request;

[0354] The determining unit 1330 is used to determine the target icon from a preset candidate icon set based on the icon insert prompt text and element attribute data;

[0355] Insertion unit 1340 is used to insert the target icon into the target collaboration canvas.

[0356] Optionally, the second acquisition unit 1320 is specifically used for:

[0357] Get at least one candidate artboard element from the target collaboration artboard, and get the relevance score between each candidate artboard element and the icon insertion request;

[0358] Based on the relevance score corresponding to each candidate canvas element, at least one associated canvas element is determined from at least one candidate canvas element, and the element attribute data of at least one associated canvas element is obtained.

[0359] Optionally, the second acquisition unit 1320 is specifically used for:

[0360] Get the position of each candidate artboard element in the target collaborative artboard;

[0361] Get the target insertion position of the icon in the target collaboration artboard based on the icon insertion request;

[0362] The relevance score between each candidate canvas element and the icon insertion request is determined based on the target insertion position and the element position.

[0363] Optionally, the second acquisition unit 1320 is specifically used for:

[0364] Based on the icon insertion request, obtain the target insertion position and the insertion object of the icon in the target collaboration artboard;

[0365] Determine the corresponding canvas element of the inserted object from at least one candidate canvas element;

[0366] The relevance score between each candidate canvas element and the icon insertion request is determined based on the canvas element of the same object, the target insertion position, and the element position.

[0367] Optionally, the determining unit 1330 is specifically used for:

[0368] Input the element attribute data into a preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set;

[0369] At least one second candidate icon is determined from a preset set of candidate icons based on the insertion probability;

[0370] The target icon is determined from at least one second candidate icon based on the icon-insertion tooltip text.

[0371] Optionally, the determining unit 1330 is specifically used for:

[0372] Feature extraction is performed on the tooltip text inserted into the icon to obtain the tooltip text features;

[0373] For each second candidate icon, feature extraction is performed to obtain the candidate icon features;

[0374] The target icon is determined from at least one second candidate icon based on the features of the prompt text and the features of the candidate icon corresponding to each second candidate icon.

[0375] Optionally, the determining unit 1330 is specifically used for:

[0376] The icon insertion prompt text is input into the first preset sub-neural network, and the feature of the icon insertion prompt text is extracted to obtain the text semantic features;

[0377] The second preset sub-neural network is used to extract key features of the text semantic features to obtain the prompt text features.

[0378] Optionally, the determining unit 1330 is specifically used for:

[0379] Retrieve the historical icon records corresponding to the target collaboration artboard;

[0380] The preference coefficient for each first candidate icon is determined based on historical icon records.

[0381] Based on the preference coefficient and insertion probability corresponding to each first candidate icon, at least one second candidate icon is determined from the preset candidate icon set.

[0382] Optionally, the insertion unit 1340 is specifically used for:

[0383] Obtain the canvas layout information and object operation information of the target collaborative canvas;

[0384] Determine the target insertion position in the target collaborative artboard based on artboard layout information and object operation information;

[0385] Insert the target icon into the target insertion position on the target collaboration canvas.

[0386] Optionally, the insertion unit 1340 is specifically used for:

[0387] Based on the element attribute data corresponding to each associated canvas element, determine the degree of association between the target icon and each associated canvas element;

[0388] Based on the canvas layout information, object operation information, and the degree of association between each associated canvas element, the target insertion position is determined in the target collaborative canvas.

[0389] Optionally, the preset candidate icon set is updated in the following ways:

[0390] Get the preset update cycle;

[0391] Based on a preset update cycle, obtain the number of times each candidate icon in the preset candidate icon set is used, and determine the low-frequency icons in the preset candidate icon set based on the number of times each candidate icon is used.

[0392] Retrieve newly added icons from the target icon library during a preset update cycle, and update the preset candidate icon set based on the newly added icons and low-frequency icons.

[0393] Reference Figure 14 , Figure 14 To implement the structural block diagram of a portion of the terminal 140 in this embodiment, the terminal includes: a radio frequency (RF) circuit 1410, a memory 1415, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490, among other components. Those skilled in the art will understand that... Figure 14The terminal 140 structure shown does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0394] The RF circuit 1410 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1480; in addition, it transmits uplink data to the base station.

[0395] The memory 1415 can be used to store software programs and modules. The processor 1480 executes various functional applications of the content terminal and the insertion of icons on the collaborative drawing board by running the software programs and modules stored in the memory 1415.

[0396] The input unit 1430 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the content terminal. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432.

[0397] Display unit 1440 can be used to display input or provided information, as well as various menus of the content terminal. Display unit 1440 may include display panel 1441.

[0398] Audio circuitry 1460, speaker 1461, and microphone 1462 provide an audio interface.

[0399] In this embodiment, the processor 1480 included in the object terminal 140 can execute the icon insertion method of the collaborative drawing board in the previous embodiment.

[0400] The target terminal 140 in this disclosure includes, but is not limited to, mobile phones, computers, intelligent voice exchange devices, smart home appliances, vehicle terminals, and aircraft. This invention can be applied to various scenarios, including but not limited to group office work and collaborative drawing.

[0401] Figure 15This is a partial structural block diagram of a server 110 implementing an embodiment of the present disclosure. The server 110 can vary significantly due to different configurations or performance characteristics, and may include one or more central processing units (CPUs) 1522 (e.g., one or more processors) and memory 1532, and one or more storage media 1530 (e.g., one or more mass storage devices) for storing application programs 1542 or data 1544. The memory 1532 and storage media 1530 may be temporary or persistent storage. The program stored in the storage media 1530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1522 may be configured to communicate with the storage media 1530 and execute the series of instruction operations in the storage media 1530 on the server.

[0402] Server 110 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input / output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0403] The central processing unit 1522 in server 110 can be used to execute the icon insertion method for the collaborative canvas according to embodiments of this disclosure.

[0404] This disclosure also provides a computer-readable storage medium for storing program code for executing the icon insertion method for the collaborative drawing board in the foregoing embodiments.

[0405] This disclosure also provides a computer program product comprising a computer program. The processor of an electronic device reads and executes the computer program, causing the electronic device to perform the icon insertion method for the collaborative drawing board described above.

[0406] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar terms and are not necessarily used to describe a particular order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0407] It should be understood that in this disclosure, "at least one item" refers to one or more items, and "more than one item" refers to two or more items. "And / or" is used to describe the relationship between related content, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related content are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0408] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0409] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0410] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0411] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0412] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0413] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0414] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0415] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for inserting icons into a collaborative canvas, characterized in that, include: Obtain the icon insertion request for inserting an icon into the target collaboration artboard, and extract the icon insertion prompt text from the icon insertion request; Based on the icon insertion request, obtain the element attribute data of at least one associated canvas element in the target collaborative canvas; Based on the icon, insert the prompt text and the element attribute data to determine the target icon from the preset candidate icon set; Insert the target icon into the target collaboration canvas.

2. The method according to claim 1, characterized in that, The step of obtaining element attribute data of at least one associated artboard element in the target collaboration artboard based on the icon insertion request includes: Obtain at least one candidate canvas element from the target collaborative canvas, and obtain the correlation score between each candidate canvas element and the icon insertion request; Based on the correlation score corresponding to each candidate canvas element, at least one associated canvas element is determined from at least one candidate canvas element, and the element attribute data of at least one associated canvas element is obtained.

3. The method according to claim 2, characterized in that, The step of obtaining the correlation score between each candidate canvas element and the icon insertion request includes: Obtain the element position of each candidate artboard element in the target collaborative artboard; Based on the icon insertion request, obtain the target insertion position for inserting the icon in the target collaboration canvas; Based on the target insertion position and the element position, a correlation score is determined between each candidate canvas element and the icon insertion request.

4. The method according to claim 3, characterized in that, The step of obtaining the target insertion position for inserting the icon in the target collaboration canvas based on the icon insertion request includes: Based on the icon insertion request, obtain the target insertion position and the insertion object of the icon in the target collaboration canvas; The process of determining the relevance score between each candidate canvas element and the icon insertion request based on the target insertion position and the element position includes: Determine the same object canvas element corresponding to the inserted object from at least one of the candidate canvas elements; Based on the same canvas element, the target insertion position, and the element position, a correlation score is determined between each candidate canvas element and the icon insertion request.

5. The method according to claim 1, characterized in that, The step of determining the target icon from a preset candidate icon set based on the inserted prompt text and the element attribute data includes: The element attribute data is input into a preset neural network model to obtain the insertion probability of each first candidate icon in the preset candidate icon set. At least one second candidate icon is determined from the preset candidate icon set based on the insertion probability; The target icon is determined from the at least one second candidate icon based on the inserted prompt text.

6. The method according to claim 5, characterized in that, The step of determining the target icon from the at least one second candidate icon based on the inserted prompt text includes: Feature extraction is performed on the inserted prompt text of the icon to obtain the prompt text features; For each second candidate icon, feature extraction is performed to obtain the candidate icon features; Based on the features of the prompt text and the features of the candidate icons corresponding to each second candidate icon, the target icon is determined from the at least one second candidate icon.

7. The method according to claim 6, characterized in that, The step of extracting features from the inserted prompt text on the icon to obtain prompt text features includes: The icon insertion prompt text is input into a first preset sub-neural network, and feature extraction is performed on the icon insertion prompt text to obtain text semantic features; The second preset sub-neural network is used to extract key features from the semantic features of the text to obtain the prompt text features.

8. The method according to claim 5, characterized in that, The step of determining at least one second candidate icon from the preset candidate icon set based on the insertion probability includes: Retrieve the historical icon records corresponding to the target collaboration canvas; Based on the historical icon records, determine the preference coefficient corresponding to each of the first candidate icons; Based on the preference coefficient and the insertion probability corresponding to each first candidate icon, at least one second candidate icon is determined from the preset candidate icon set.

9. The method according to claim 1, characterized in that, The step of inserting the target icon into the target collaboration canvas includes: Obtain the canvas layout information and object operation information of the target collaborative canvas; Based on the canvas layout information and the object operation information, the target insertion position is determined in the target collaborative canvas; Insert the target icon into the target insertion position on the target collaboration canvas.

10. The method according to claim 9, characterized in that, Determining the target insertion position in the target collaborative artboard based on the artboard layout information and the object operation information includes: Based on the element attribute data corresponding to each of the associated canvas elements, the degree of association between the target icon and each of the associated canvas elements is determined; Based on the canvas layout information, the object operation information, and the degree of association corresponding to each associated canvas element, the target insertion position is determined in the target collaborative canvas.

11. The method according to claim 1, characterized in that, The preset candidate icon set is updated in the following ways: Get the preset update cycle; Based on the preset update cycle, obtain the number of times each candidate icon in the preset candidate icon set is used, and determine the low-frequency icons in the preset candidate icon set based on the number of times each candidate icon is used. Obtain newly added icons from the target icon library during the preset update cycle, and update the preset candidate icon set based on the newly added icons and the low-frequency icons.

12. An icon insertion device for a collaborative drawing board, characterized in that, include: The first acquisition unit is used to acquire an icon insertion request for inserting an icon into the target collaborative canvas, and extract icon insertion prompt text from the icon insertion request. The second acquisition unit is used to acquire element attribute data of at least one associated canvas element in the target collaborative canvas based on the icon insertion request; The determining unit is used to determine the target icon from a preset candidate icon set based on the inserted prompt text of the icon and the element attribute data; An insertion unit is used to insert the target icon into the target collaboration canvas.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the icon insertion method for the collaborative drawing board according to any one of claims 1 to 11.

14. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the icon insertion method for the collaborative drawing board according to any one of claims 1 to 11.

15. A computer program product comprising a computer program that is read and executed by a processor of an electronic device, causing the electronic device to perform the icon insertion method for a collaborative drawing board according to any one of claims 1 to 11.

Images