Element selection method, device, storage medium and apparatus based on mobile operation

Abstract

The present invention discloses an element selection method, equipment, storage medium and device based on movement operation. When receiving an element selection instruction triggered by the user in the current interface, the method detects the cursor movement operation input by the user based on the current interface, determines the corresponding cursor movement trajectory according to the cursor movement operation, generates a closed area based on the cursor movement trajectory, judges whether there are interface elements in the current interface within the closed area, and if there are interface elements in the current interface within the closed area, selects the interface elements in the closed area, and the closed area moved out according to the required operation can adapt to the concentrated and complex arrangement of the relationship diagram, thereby circling the required interface elements and improving the accuracy of interface element selection.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an element selection method, equipment, storage medium and device based on move operation. Background Art

[0002] Currently, the selection method for elements in the interface is a rectangular frame, which is used to select the required interface elements. However, the relationship diagram is relatively concentrated and complex, and the selection method using a rectangular frame cannot meet the needs. It is easy to select unnecessary elements into the rectangular frame, or multiple selections are required to select the required elements. The selection accuracy of interface elements is low.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide an element selection method, device, storage medium and apparatus based on mobile operation, aiming to solve the technical problem of low accuracy of frame selection of interface elements in the prior art.

[0005] To achieve the above object, the present invention provides an element selection method based on a move operation, and the element selection method based on a move operation comprises the following steps:

[0006] When receiving an element selection instruction triggered by the user in the current interface, detecting a cursor movement operation input by the user based on the current interface;

[0007] Determine a corresponding cursor movement trajectory according to the cursor movement operation;

[0008] Generate a closed area based on the cursor movement trajectory;

[0009] Determine whether there is an interface element in the current interface that is within the closed area;

[0010] If there is an interface element in the closed area in the current interface, the interface element in the closed area is selected.

[0011] Preferably, the determining whether there is an interface element in the current interface within the closed area specifically includes:

[0012] Traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements;

[0013] Get the center point of the current interface element;

[0014] Generate a target ray along a preset direction with the center point as the starting point;

[0015] Obtaining the number of intersections between the target ray and the edge of the closed area;

[0016] It is determined whether the current interface element is within the closed area according to the number of intersections.

[0017] Preferably, judging whether the current interface element is within the closed area according to the number of intersections specifically includes:

[0018] Determine whether the number of intersection points is an odd number;

[0019] If the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

[0020] Preferably, the determining whether there is an interface element in the current interface within the closed area specifically includes:

[0021] Traversing the interface elements in the current interface, and taking the traversed interface elements as the interface elements to be confirmed;

[0022] Obtaining the element area of ​​the interface element to be confirmed;

[0023] Calculating the area ratio of the interface element to be confirmed within the closed area according to the element area;

[0024] It is determined whether the interface element to be confirmed is within the closed area according to the area ratio.

[0025] Preferably, judging whether the to-be-confirmed interface element is within the closed area according to the area ratio specifically includes:

[0026] Determining whether the area ratio is greater than a first preset ratio;

[0027] If the area ratio is greater than the first preset ratio, it is determined that the interface element to be confirmed is within the closed area.

[0028] Preferably, if the area ratio is greater than the first preset ratio, after determining that the interface element to be confirmed is within the closed area, the element selection method based on the moving operation further includes:

[0029] Determine whether there is an interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than a second preset ratio, and the second preset ratio is less than the first preset ratio;

[0030] If so, the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio is identified as the element to be identified;

[0031] Performing text recognition on the element to be recognized to obtain text information to be recognized;

[0032] Performing text recognition on the interface elements within the closed area to obtain target text information;

[0033] It is determined whether the element to be identified is within the closed area according to the text information to be identified and the target text information.

[0034] Preferably, judging whether the element to be identified is within the closed area according to the text information to be identified and the target text information specifically includes:

[0035] Calculating the text similarity between the to-be-recognized text information and the target text information;

[0036] According to the text similarity, it is determined whether the element to be identified is within the closed area.

[0037] Preferably, the calculating of the text similarity between the to-be-recognized text information and the target text information specifically includes:

[0038] Extracting keywords from the text information to be identified to obtain keywords of the text to be identified;

[0039] Extracting keywords from the target text information to obtain keywords of the target text;

[0040] The text similarity between the text information to be recognized and the target text information is calculated according to the keywords of the text to be recognized and the keywords of the target text.

[0041] Preferably, there are multiple interface elements within the closed area;

[0042] The determining, according to the to-be-recognized text information and the target text information, whether the to-be-recognized element is within the closed area specifically includes:

[0043] Clustering target text information of multiple interface elements within the closed area to obtain a cluster center;

[0044] Calculating the clustering similarity between the text information to be identified and the cluster center;

[0045] According to the cluster similarity, it is determined whether the element to be identified is within the closed area.

[0046] Preferably, judging whether the element to be identified is within the closed area according to the cluster similarity specifically includes:

[0047] Determining whether the cluster similarity exceeds a preset similarity threshold;

[0048] When the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area.

[0049] Preferably, generating a closed area based on the cursor movement trajectory specifically includes:

[0050] Determine whether there is a mutation point in the cursor movement trajectory;

[0051] If there is a mutation point in the cursor movement trajectory, taking the coordinate points in the cursor movement trajectory other than the mutation point as the target coordinate point;

[0052] Connect the target coordinate points to obtain a closed area.

[0053] Preferably, the determining whether there is a mutation point in the cursor movement trajectory specifically includes:

[0054] Calculating the coordinate difference between adjacent coordinate points in the cursor movement trajectory;

[0055] According to the coordinate difference, it is determined whether there is a mutation point in the cursor movement trajectory.

[0056] Preferably, judging whether there is a mutation point in the cursor movement trajectory according to the coordinate difference specifically includes:

[0057] Arrange the coordinate differences in descending order to obtain a sorted list;

[0058] Obtaining the median of the sorted list;

[0059] Calculate the median difference between each coordinate difference and the median;

[0060] Determining whether the median difference exceeds a preset difference threshold;

[0061] If the median difference exceeds a preset difference threshold, the coordinate point corresponding to the median difference is identified as a mutation point in the cursor movement trajectory.

[0062] In addition, to achieve the above-mentioned purpose, the present invention also proposes an element selection device based on mobile operation, wherein the element selection device based on mobile operation includes a memory, a processor, and an element selection program based on mobile operation stored in the memory and executable on the processor, wherein the element selection program based on mobile operation is configured to implement the steps of the element selection method based on mobile operation as described above.

[0063] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which is stored an element selection program based on move operation, and when the element selection program based on move operation is executed by a processor, the steps of the element selection method based on move operation as described above are implemented.

[0064] In addition, to achieve the above-mentioned purpose, the present invention further proposes an element selection device based on a moving operation, and the element selection device based on a moving operation comprises:

[0065] A detection module, configured to detect a cursor movement operation input by a user based on the current interface when receiving an element selection instruction triggered by the user in the current interface;

[0066] A determination module, used to determine a corresponding cursor movement trajectory according to the cursor movement operation;

[0067] A generating module, used for generating a closed area based on the cursor movement trajectory;

[0068] A judging module, used for judging whether there is an interface element in the current interface that is within the closed area;

[0069] The selection module is used to select the interface element within the closed area if there is an interface element within the closed area in the current interface.

[0070] Preferably, the judgment module is also used to traverse the interface elements in the current interface and use the traversed interface elements as the current interface elements; obtain the center point of the current interface element; generate a target ray with the center point as the starting point along a preset direction; obtain the number of intersections between the target ray and the edge of the closed area; and judge whether the current interface element is within the closed area based on the number of intersections.

[0071] Preferably, the judgment module is further used to judge whether the number of intersections is an odd number; if the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

[0072] Preferably, the judgment module is also used to traverse the interface elements in the current interface and use the traversed interface elements as the interface elements to be confirmed; obtain the element area of ​​the interface element to be confirmed; calculate the area ratio of the interface element to be confirmed in the closed area based on the element area; and judge whether the interface element to be confirmed is in the closed area based on the area ratio.

[0073] Preferably, the generating module is further used to determine whether there is a mutation point in the cursor movement trajectory; if there is a mutation point in the cursor movement trajectory, the coordinate points in the cursor movement trajectory other than the mutation point are used as target coordinate points; and the target coordinate points are connected to obtain a closed area.

[0074] In the present invention, when an element selection instruction triggered by the user in the current interface is received, the cursor movement operation input by the user based on the current interface is detected, the corresponding cursor movement trajectory is determined according to the cursor movement operation, and a closed area is generated based on the cursor movement trajectory. The cursor can be moved at will as needed, so as to adapt to complex relationship diagram arrangements and select any element; it is determined whether there are interface elements in the closed area in the current interface. If there are interface elements in the closed area in the current interface, the interface elements in the closed area are selected. The closed area moved out according to the required operation can adapt to the concentrated and complex relationship diagram arrangements, so as to circle the required interface elements and improve the accuracy of interface element selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a structural diagram of an element selection device based on mobile operation in a hardware operating environment involved in an embodiment of the present invention;

[0076] Figure 2 It is a flow chart of a first embodiment of the element selection method based on the moving operation of the present invention;

[0077] Figure 3 A schematic diagram of the relationship between interface elements and cursor movement trajectory;

[0078] Figure 4 It is a schematic diagram of the relationship between the interface elements and the closed area;

[0079] Figure 5 It is a flow chart of a second embodiment of the element selection method based on the moving operation of the present invention;

[0080] Figure 6 It is a flow chart of a third embodiment of the element selection method based on the moving operation of the present invention;

[0081] Figure 7 It is a flow chart of a fourth embodiment of the element selection method based on the moving operation of the present invention;

[0082] Figure 8 It is a structural block diagram of the first embodiment of the element selection device based on the moving operation of the present invention.

[0083] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0084] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0085] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of a device structure for selecting elements based on mobile operations in a hardware operating environment according to an embodiment of the present invention.

[0086] like Figure 1 As shown, the element selection device based on mobile operation may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable memory (Non-volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0087] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the element selection device based on the movement operation, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0088] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an element selection program based on mobile operations.

[0089] exist Figure 1In the element selection device based on mobile operation shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the user device; the element selection device based on mobile operation calls the element selection program based on mobile operation stored in the memory 1005 through the processor 1001, and executes the element selection method based on mobile operation provided by the embodiment of the present invention.

[0090] The element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and performs the following operations:

[0091] When receiving an element selection instruction triggered by the user in the current interface, detecting a cursor movement operation input by the user based on the current interface;

[0092] Determine a corresponding cursor movement trajectory according to the cursor movement operation;

[0093] Generate a closed area based on the cursor movement trajectory;

[0094] Determine whether there is an interface element in the current interface that is within the closed area;

[0095] If there is an interface element in the closed area in the current interface, the interface element in the closed area is selected.

[0096] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0097] Traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements;

[0098] Get the center point of the current interface element;

[0099] Generate a target ray along a preset direction with the center point as the starting point;

[0100] Obtaining the number of intersections between the target ray and the edge of the closed area;

[0101] It is determined whether the current interface element is within the closed area according to the number of intersections.

[0102] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0103] Determine whether the number of intersection points is an odd number;

[0104] If the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

[0105] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0106] Traversing the interface elements in the current interface, and taking the traversed interface elements as the interface elements to be confirmed;

[0107] Obtaining the element area of ​​the interface element to be confirmed;

[0108] Calculating the area ratio of the interface element to be confirmed within the closed area according to the element area;

[0109] It is determined whether the interface element to be confirmed is within the closed area according to the area ratio.

[0110] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0111] Determining whether the area ratio is greater than a first preset ratio;

[0112] If the area ratio is greater than the first preset ratio, it is determined that the interface element to be confirmed is within the closed area.

[0113] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0114] Determine whether there is an interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than a second preset ratio, and the second preset ratio is less than the first preset ratio;

[0115] If so, the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio is identified as the element to be identified;

[0116] Performing text recognition on the element to be recognized to obtain text information to be recognized;

[0117] Performing text recognition on the interface elements within the closed area to obtain target text information;

[0118] It is determined whether the element to be identified is within the closed area according to the text information to be identified and the target text information.

[0119] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0120] Calculating the text similarity between the to-be-recognized text information and the target text information;

[0121] According to the text similarity, it is determined whether the element to be identified is within the closed area.

[0122] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0123] Extracting keywords from the text information to be identified to obtain keywords of the text to be identified;

[0124] Extracting keywords from the target text information to obtain keywords of the target text;

[0125] The text similarity between the text information to be recognized and the target text information is calculated according to the keywords of the text to be recognized and the keywords of the target text.

[0126] Furthermore, there are multiple interface elements in the closed area, and the element selection device based on movement operation calls the element selection program based on movement operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0127] Clustering target text information of multiple interface elements within the closed area to obtain a cluster center;

[0128] Calculating the clustering similarity between the text information to be identified and the cluster center;

[0129] According to the cluster similarity, it is determined whether the element to be identified is within the closed area.

[0130] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0131] Determining whether the cluster similarity exceeds a preset similarity threshold;

[0132] When the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area.

[0133] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0134] Determine whether there is a mutation point in the cursor movement trajectory;

[0135] If there is a mutation point in the cursor movement trajectory, taking the coordinate points in the cursor movement trajectory other than the mutation point as the target coordinate point;

[0136] Connect the target coordinate points to obtain a closed area.

[0137] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0138] Calculating the coordinate difference between adjacent coordinate points in the cursor movement trajectory;

[0139] According to the coordinate difference, it is determined whether there is a mutation point in the cursor movement trajectory.

[0140] Furthermore, the element selection device based on the moving operation calls the element selection program based on the moving operation stored in the memory 1005 through the processor 1001, and further performs the following operations:

[0141] Arrange the coordinate differences in descending order to obtain a sorted list;

[0142] Obtaining the median of the sorted list;

[0143] Calculate the median difference between each coordinate difference and the median;

[0144] Determining whether the median difference exceeds a preset difference threshold;

[0145] If the median difference exceeds a preset difference threshold, the coordinate point corresponding to the median difference is identified as a mutation point in the cursor movement trajectory.

[0146] In this embodiment, when an element selection instruction triggered by the user in the current interface is received, the cursor movement operation input by the user based on the current interface is detected, the corresponding cursor movement trajectory is determined according to the cursor movement operation, and a closed area is generated based on the cursor movement trajectory. The cursor can be moved at will as needed, so as to adapt to complex relationship diagram arrangements and select any element; it is determined whether there are interface elements in the closed area in the current interface. If there are interface elements in the closed area in the current interface, the interface elements in the closed area are selected. The closed area moved out according to the required operation can adapt to the concentrated and complex arrangement of the relationship diagram, thereby circling the required interface elements and improving the accuracy of interface element selection.

[0147] Based on the above hardware structure, an embodiment of the element selection method based on the moving operation of the present invention is proposed.

[0148] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the element selection method based on the moving operation of the present invention is provided.

[0149] In a first embodiment, the element selection method based on the move operation includes the following steps:

[0150] Step S10: When an element selection instruction triggered by the user in the current interface is received, a cursor movement operation input by the user based on the current interface is detected.

[0151] It should be understood that the executor of this embodiment is the element selection device based on mobile operation, and the element selection device based on mobile operation can be an electronic device such as a smart phone, a tablet computer or a personal computer, and this embodiment does not limit this. When the user clicks on the current interface, it can be considered that the element selection instruction triggered by the user in the current interface is received, and the cursor is continuously moved, that is, the cursor movement operation input based on the current interface. Taking a personal computer as an example, the user presses the mouse to select a starting point and records the starting point. In the process where the user does not release the mouse and moves, that is, the cursor movement operation input by the user based on the current interface, the moving coordinate points of the cursor are recorded in real time.

[0152] Step S20: determining a corresponding cursor movement trajectory according to the cursor movement operation.

[0153] It is understandable that the cursor coordinate points in the process of the cursor movement operation are recorded, and the cursor coordinate points recorded in the movement operation are connected in sequence to obtain the cursor movement trajectory. Figure 3 As shown, Figure 3 Schematic diagram of the relationship between interface elements and cursor movement trajectory.

[0154] Step S30: generating a closed area based on the cursor movement trajectory.

[0155] It should be noted that the cursor movement trajectory is connected into a closed area by connecting the coordinate points that the cursor passes through during the movement from the starting point one by one until the end point is connected to form a closed area. The closed area is usually an irregular polygon. If the cursor movement trajectory itself is a closed area, the cursor movement trajectory is directly used as the closed area; if the cursor movement trajectory is not a closed area, and the coordinates of the starting point and the end point are different, the coordinate points that the cursor passes through during the movement are connected from the starting point one by one until the end point is connected, and the end point and the starting point are also connected in a straight line, thereby generating the closed area. Figure 4 As shown, Figure 4 Schematic diagram of the relationship between interface elements and closed areas.

[0156] Step S40: Determine whether there is an interface element in the current interface that is within the closed area.

[0157] In a specific implementation, the current interface is a display page including multiple interface elements, which may be a relationship diagram with complex relationships, or other page content. To determine whether the closed area selects the interface elements in the current interface, specifically, the interface elements in the current interface may be traversed, and the traversed interface elements may be used as the current interface elements. The center point of the current interface element may be obtained, and a target ray may be generated along a preset direction with the center point as the starting point. The preset direction includes a horizontal direction. If the number of intersections between the target ray and the edge of the closed area is an odd number, the current interface element is deemed to be an interface element within the closed area. Figure 4 As shown, the number of intersections between the center points of interface element 2 and interface element 5 and the edges of the closed area is 1, which is an odd number, and interface element 2 and interface element 5 are considered to be interface elements within the closed area.

[0158] It is understandable that the element area of ​​the current interface element can also be obtained, the area ratio of the current interface element in the closed area is calculated, and it is determined whether the area ratio exceeds a first preset ratio. The first preset ratio is usually set to 0.5, that is, when the area ratio exceeds half of the element area, the current interface element is identified as an interface element in the closed area. Figure 4 As shown, if the area proportions corresponding to interface element 2 and interface element 5 exceed the first preset proportion, interface element 2 and interface element 5 are determined to be interface elements within the closed area.

[0159] Step S50: If there is an interface element in the closed area in the current interface, the interface element in the closed area is selected.

[0160] It should be understood that if there is an interface element in the closed area in the current interface, the interface element in the closed area is considered to be the interface element that the user needs to select, and it is selected to perform the next operation. Figure 4 As shown, if interface element 2 and interface element 5 are identified as interface elements that the user needs to select, then interface element 2 and interface element 5 are selected.

[0161] In this embodiment, when an element selection instruction triggered by the user in the current interface is received, the cursor movement operation input by the user based on the current interface is detected, the corresponding cursor movement trajectory is determined according to the cursor movement operation, and a closed area is generated based on the cursor movement trajectory. The cursor can be moved at will as needed, so as to adapt to complex relationship diagram arrangements and select any element; it is determined whether there are interface elements in the closed area in the current interface. If there are interface elements in the closed area in the current interface, the interface elements in the closed area are selected. The closed area moved out according to the required operation can adapt to the concentrated and complex arrangement of the relationship diagram, thereby circling the required interface elements and improving the accuracy of interface element selection.

[0162] Reference Figure 5 , Figure 5 This is a flow chart of a second embodiment of the element selection method based on the moving operation of the present invention. Figure 2 The first embodiment shown provides a second embodiment of the element selection method based on the moving operation of the present invention.

[0163] In the second embodiment, the step S40 includes:

[0164] Step S401: traverse the interface elements in the current interface, and use the traversed interface elements as the current interface elements.

[0165] It should be understood that there are usually multiple interface elements in the current interface. Taking the current interface as a relationship graph as an example, the interface elements are the nodes in the relationship graph. Each node in the relationship graph is traversed and the traversed nodes are used as the current interface elements.

[0166] Step S402: Acquire the center point of the current interface element.

[0167] It is understandable that the center point of the multiple deformations formed by the boundary of the current interface element is the center point of the current interface element. The current interface element is usually presented in the form of a text box, and the center point of the text box where the current interface element is located can be used as the center point of the current interface element.

[0168] Step S403: Generate a target ray along a preset direction with the center point as the starting point.

[0169] It should be noted that the preset direction includes a horizontal direction, a vertical direction or an arbitrary direction. A target ray is generated along the preset direction with the center point as the starting point. If the center point is within the closed area, the number of intersections between the target ray and the closed area is an odd number.

[0170] Step S404: Obtain the number of intersections between the target ray and the edge of the closed area.

[0171] In a specific implementation, the closed area is usually an irregular polygon, and the edge of the irregular polygon is obtained, and it is determined whether the target ray intersects with the edge of the irregular polygon to obtain the number of intersection points.

[0172] Step S405: determining whether the current interface element is within the closed area according to the number of intersections.

[0173] It should be understood that if the center point is within the closed area, the number of intersections between the target ray and the closed area is an odd number; if the center point is not within the closed area, the number of intersections between the target ray and the closed area is an even number. In this embodiment, step S405 includes: determining whether the number of intersections is an odd number; if the number of intersections is an odd number, determining that the current interface element is within the closed area.

[0174] In this embodiment, the interface elements in the current interface are traversed, the traversed interface elements are used as the current interface elements, the center point of the current interface element is obtained, and a target ray is generated along a preset direction with the center point as the starting point, and the number of intersections between the target ray and the edge of the closed area is obtained. It is determined whether the current interface element is within the closed area based on the number of intersections, thereby improving the accuracy of interface element selection.

[0175] Reference Figure 6 , Figure 6 This is a flow chart of a third embodiment of the element selection method based on the moving operation of the present invention. Figure 2 The first embodiment shown is a third embodiment of the element selection method based on the moving operation of the present invention.

[0176] In the third embodiment, the step S40 includes:

[0177] Step S406: traverse the interface elements in the current interface, and use the traversed interface elements as interface elements to be confirmed.

[0178] It should be understood that there are usually multiple interface elements in the current interface. Taking the current interface as a relationship graph as an example, the interface elements are the nodes in the relationship graph. Each node in the relationship graph is traversed and the traversed nodes are used as the interface elements to be confirmed.

[0179] Step S407: Acquire the element area of ​​the interface element to be confirmed.

[0180] It is understandable that the area of ​​the multiple deformations formed by the boundary of the current interface element is the element area of ​​the current interface element. The current interface element is usually presented in the form of a text box, and the area of ​​the text box where the current interface element is located can be used as the element area of ​​the current interface element.

[0181] Step S408: Calculate the area ratio of the to-be-confirmed interface element within the closed area according to the element area.

[0182] It should be noted that the area of ​​the to-be-confirmed interface element in the closed area is obtained, and the area ratio of the to-be-confirmed interface element in the closed area is calculated according to the area of ​​the to-be-confirmed interface element and the area of ​​the element. That is, the ratio between the area of ​​the to-be-confirmed interface element and the area of ​​the element is calculated, and the ratio is the area ratio.

[0183] Step S409: determining whether the interface element to be confirmed is within the closed area according to the area ratio.

[0184] In a specific implementation, if the area ratio exceeds a first preset ratio, the interface element to be confirmed is determined to be within the closed area. The first preset ratio is usually set to 0.5, that is, when the area ratio exceeds half of the element area, the current interface element is determined to be an interface element within the closed area. Big data analysis can also be performed to set the first preset ratio based on a large number of frame selection samples, which is not limited in this embodiment. In this embodiment, step S409 includes: determining whether the area ratio is greater than the first preset ratio; if the area ratio is greater than the first preset ratio, determining that the interface element to be confirmed is within the closed area.

[0185] Further, in this embodiment, if the area ratio is greater than the first preset ratio, after determining that the interface element to be confirmed is within the closed area, the method further includes:

[0186] Determine whether there is an interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than a second preset ratio, and the second preset ratio is less than the first preset ratio; if so, identify the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio as an element to be identified; perform text recognition on the element to be identified to obtain text information to be identified; perform text recognition on the interface element within the closed area to obtain target text information; and determine whether the element to be identified is within the closed area based on the text information to be identified and the target text information.

[0187] It should be understood that the area ratio of the to-be-confirmed interface element is less than or equal to the first preset ratio and greater than the second preset ratio, that is, part of it is within the closed area and the other part is outside the closed area, and the second preset ratio is smaller than the first preset ratio. For example, the first preset ratio is 0.5, and the second preset ratio is 0.3. The second preset ratio can be set according to empirical values, or by performing big data analysis and setting the second preset ratio based on a large number of box-selected samples. This embodiment does not impose any restrictions on this.

[0188] It can be understood that if there is an interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio, then the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio will be identified as an element to be identified, and it is necessary to further determine whether the element to be identified is the interface element that the user wants to select.

[0189] It should be noted that the content of the element to be identified can be identified, the identification text information corresponding to the element to be identified can be obtained, and then the interface element in the closed area can be obtained, the text of the interface element in the closed area can be identified, the target text information corresponding to the interface element in the closed area can be obtained, the text similarity between the identification text information and the target text information can be calculated, and it is judged whether the text similarity exceeds the first similarity threshold. The first similarity threshold can be set according to the experience value. For example, the first similarity threshold is set to 50%. If the text similarity exceeds the first similarity threshold, it is determined that the element to be identified is in the closed area and the element to be identified is selected; if the text similarity does not exceed the first similarity threshold, it is determined that the element to be identified is outside the closed area. In this embodiment, judging whether the element to be identified is in the closed area according to the text information to be identified and the target text information specifically includes: calculating the text similarity between the text information to be identified and the target text information; judging whether the element to be identified is in the closed area according to the text similarity.

[0190] Furthermore, in this embodiment, the calculating of the text similarity between the to-be-recognized text information and the target text information specifically includes:

[0191] Keyword extraction is performed on the text information to be identified to obtain keywords of the text to be identified; keyword extraction is performed on the target text information to obtain keywords of the target text; and the text similarity between the text information to be identified and the target text information is calculated based on the keywords of the text to be identified and the keywords of the target text.

[0192] In a specific implementation, the text information to be identified is segmented to obtain all the first words of the text information to be identified, and useless words in the first words are filtered to obtain the text keywords to be identified, wherein the useless words include modal particles and auxiliary words and other words without practical meaning. The target text information is segmented to obtain all the second words of the target text information, and useless words in the second words are filtered to obtain the target text keywords. The TF-IDF value, TF term frequency, and IDF inverse document frequency of the text keywords to be identified are calculated, and the text information to be identified is represented as a first word vector composed of the text keywords to be identified and the corresponding TF-IDF value; the TF-IDF value of the target text keywords is calculated, and the target text information is represented as a second word vector composed of the target text keywords and the corresponding TF-IDF value; the cosine distance between the first word vector and the second word vector is calculated, and the cosine distance is used as the text similarity between the text information to be identified and the target text information.

[0193] Further, in this embodiment, there are multiple interface elements in the closed area;

[0194] The determining, according to the to-be-recognized text information and the target text information, whether the to-be-recognized element is within the closed area specifically includes:

[0195] Clustering target text information of multiple interface elements within the closed area to obtain cluster centers; calculating cluster similarity between the text information to be identified and the cluster centers; and judging whether the element to be identified is within the closed area based on the cluster similarity.

[0196] It should be understood that the target text information of multiple interface elements in the closed area can be clustered by the affinity propagation clustering (AP) algorithm, which performs clustering based on the similarity between data points and can be symmetrical or asymmetrical. The affinity propagation clustering algorithm does not need to determine the number of clusters first, but regards all data points as potential cluster centers.

[0197] It can be understood that the target similarity between each target text information is calculated, and a similarity matrix can be constructed according to each target similarity, with each target text information as a node and the target similarity between each target text information as the value of the matrix, and then clustering is performed through the AP algorithm to obtain a cluster cluster, wherein the cluster cluster includes a cluster center, and the cluster center has the strongest similarity with other target text information in the corresponding cluster cluster and is most representative of the cluster cluster.

[0198] It should be understood that the cluster similarity between the text information to be identified and the cluster center is calculated to determine whether the cluster similarity exceeds a preset similarity threshold. The preset similarity threshold can be set according to an empirical value. For example, the preset similarity threshold is set to 50%. Big data analysis can also be performed to set the preset similarity threshold based on a large number of framed samples. This is not limited in this embodiment. When the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area; when the cluster similarity does not exceed the preset similarity threshold, it is determined that the element to be identified is outside the closed area. In this embodiment, judging whether the element to be identified is within the closed area based on the cluster similarity specifically includes: judging whether the cluster similarity exceeds the preset similarity threshold; when the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area.

[0199] In this embodiment, by traversing the interface elements in the current interface, the traversed interface elements are used as interface elements to be confirmed, and the element area of ​​the interface elements to be confirmed is obtained. According to the element area, the area ratio of the interface elements to be confirmed in the closed area is calculated, and whether the interface elements to be confirmed are in the closed area is judged according to the area ratio. Interface elements whose area ratio exceeds a certain proportion are identified as interface elements in the closed area, thereby solving the problem of whether partially selected interface elements are selected, and improving the efficiency and accuracy of selecting interface elements.

[0200] Reference Figure 7 , Figure 7 This is a flow chart of the third embodiment of the method for selecting an element based on a moving operation of the present invention. Based on the first embodiment, the second embodiment or the third embodiment of the method, the fourth embodiment of the method for selecting an element based on a moving operation of the present invention is proposed. In this embodiment, the first embodiment is used as the basis for description.

[0201] In this embodiment, the step S30 includes:

[0202] Step S301: Determine whether there is a mutation point in the cursor movement trajectory.

[0203] It should be understood that the cursor coordinate points in the process of the cursor movement operation are recorded, and the cursor coordinate points recorded in the movement operation are connected in sequence to obtain the cursor movement trajectory. The user inputs a cursor movement operation based on the current interface. If the cursor deviates from the normal trajectory due to external factors during the cursor movement operation, the mutation point will appear. For example, when the user moves the cursor with the mouse, the arm is hit by external force or other reasons cause the cursor to move beyond the original trajectory, and the mutation point will appear.

[0204] Step S302: If there is a mutation point in the cursor movement trajectory, the coordinate points in the cursor movement trajectory other than the mutation point are used as target coordinate points.

[0205] It is understandable that if there is a mutation point in the cursor movement trajectory, the mutation point needs to be filtered, and the coordinate points in the cursor movement trajectory other than the mutation point are taken as target coordinate points and retained.

[0206] Step S303: Connect the target coordinate points to obtain a closed area.

[0207] It should be noted that the target coordinate points are connected in sequence according to the recorded time sequence to obtain the closed area.

[0208] Furthermore, in this embodiment, step S301 includes:

[0209] Calculate the coordinate difference between adjacent coordinate points in the cursor movement trajectory; and determine whether there is a mutation point in the cursor movement trajectory according to the coordinate difference.

[0210] In a specific implementation, each coordinate point in the cursor movement trajectory is obtained, and the coordinate difference between adjacent coordinate points is calculated. Usually, in the process of moving the cursor, the cursor will move smoothly, and the coordinate difference is usually within a certain range. A large number of sample cursor movement trajectories can be obtained for analysis to obtain the difference range of the coordinate difference in the smooth movement process, and the coordinate difference is compared with the difference range. If the coordinate difference exceeds the difference range, it is determined whether there is a mutation point in the cursor movement trajectory, and the coordinate point corresponding to the coordinate difference exceeding the difference range is identified as the mutation point. Usually, the coordinate difference between the mutation point and the adjacent coordinate point in front of it is abnormal, and the coordinate difference between the mutation point and the adjacent coordinate point behind it is also abnormal. Therefore, the coordinate point whose coordinate difference between the adjacent coordinate points in front and behind exceeds the difference range can be identified as the mutation point.

[0211] Further, in this embodiment, judging whether there is a mutation point in the cursor movement trajectory according to the coordinate difference includes:

[0212] Arrange the coordinate differences in order from large to small to obtain a sorted list; obtain the median of the sorted list; calculate the median difference between each coordinate difference and the median; determine whether the median difference exceeds a preset difference threshold; if the median difference exceeds the preset difference threshold, identify the coordinate point corresponding to the median difference as a mutation point in the cursor movement trajectory.

[0213] It should be understood that, usually in the process of moving the cursor, the cursor is moved smoothly, and the difference between the coordinate differences is not large. The coordinate differences are arranged in order from large to small to obtain a sorted list, and the median of the sorted list can reflect the normal value of the coordinate difference. That is to say, in the process of moving the cursor, the coordinate difference is usually close to the median.

[0214] It is understandable that the preset difference threshold can be set according to an empirical value, or can be analyzed according to big data, a large number of sample cursor movement trajectories are obtained, and the coordinate differences in the sample cursor movement trajectories are analyzed to set the preset difference threshold. The median difference between each coordinate difference and the median is calculated. If the coordinate point is a coordinate point generated by normal movement, the median difference between the corresponding coordinate difference and the median is smaller. By comparing the median difference with the preset difference threshold, if the median difference exceeds the preset difference threshold, the coordinate point corresponding to the median difference is identified as a mutation point in the cursor movement trajectory. If the median difference does not exceed the preset difference threshold, the coordinate point corresponding to the median difference is identified as the target coordinate point in the cursor movement trajectory. The target coordinate points can be connected in sequence according to the time order of the records to obtain the closed area.

[0215] In this embodiment, it is determined whether there is a mutation point in the cursor movement trajectory. If there is a mutation point in the cursor movement trajectory, the coordinate points in the cursor movement trajectory other than the mutation point are used as target coordinate points, and the target coordinate points are connected to obtain a closed area. By eliminating the mutation points, the accuracy of the closed area is improved, thereby providing the accuracy of the interface element selection.

[0216] In addition, an embodiment of the present invention further provides a storage medium, on which a move operation-based element selection program is stored. When the move operation-based element selection program is executed by a processor, the following steps are implemented:

[0217] When receiving an element selection instruction triggered by the user in the current interface, detecting a cursor movement operation input by the user based on the current interface;

[0218] Determine a corresponding cursor movement trajectory according to the cursor movement operation;

[0219] Generate a closed area based on the cursor movement trajectory;

[0220] Determine whether there is an interface element in the current interface that is within the closed area;

[0221] If there is an interface element in the closed area in the current interface, the interface element in the closed area is selected.

[0222] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0223] Traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements;

[0224] Get the center point of the current interface element;

[0225] Generate a target ray along a preset direction with the center point as the starting point;

[0226] Obtaining the number of intersections between the target ray and the edge of the closed area;

[0227] It is determined whether the current interface element is within the closed area according to the number of intersections.

[0228] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0229] Determine whether the number of intersection points is an odd number;

[0230] If the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

[0231] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0232] Traversing the interface elements in the current interface, and taking the traversed interface elements as the interface elements to be confirmed;

[0233] Obtaining the element area of ​​the interface element to be confirmed;

[0234] Calculating the area ratio of the interface element to be confirmed within the closed area according to the element area;

[0235] It is determined whether the interface element to be confirmed is within the closed area according to the area ratio.

[0236] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0237] Determining whether the area ratio is greater than a first preset ratio;

[0238] If the area ratio is greater than the first preset ratio, it is determined that the interface element to be confirmed is within the closed area.

[0239] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0240] Determine whether there is an interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than a second preset ratio, and the second preset ratio is less than the first preset ratio;

[0241] If so, the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio is identified as the element to be identified;

[0242] Performing text recognition on the element to be recognized to obtain text information to be recognized;

[0243] Performing text recognition on the interface elements within the closed area to obtain target text information;

[0244] It is determined whether the element to be identified is within the closed area according to the text information to be identified and the target text information.

[0245] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0246] Calculating the text similarity between the to-be-recognized text information and the target text information;

[0247] According to the text similarity, it is determined whether the element to be identified is within the closed area.

[0248] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0249] Extracting keywords from the text information to be identified to obtain keywords of the text to be identified;

[0250] Extracting keywords from the target text information to obtain keywords of the target text;

[0251] The text similarity between the text information to be recognized and the target text information is calculated according to the keywords of the text to be recognized and the keywords of the target text.

[0252] Furthermore, there are multiple interface elements in the closed area, and when the element selection program based on the moving operation is executed by the processor, the following operations are also implemented:

[0253] Clustering target text information of multiple interface elements within the closed area to obtain a cluster center;

[0254] Calculating the clustering similarity between the text information to be identified and the cluster center;

[0255] According to the cluster similarity, it is determined whether the element to be identified is within the closed area.

[0256] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0257] Determining whether the cluster similarity exceeds a preset similarity threshold;

[0258] When the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area.

[0259] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0260] Determine whether there is a mutation point in the cursor movement trajectory;

[0261] If there is a mutation point in the cursor movement trajectory, taking the coordinate points in the cursor movement trajectory other than the mutation point as the target coordinate point;

[0262] Connect the target coordinate points to obtain a closed area.

[0263] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0264] Calculating the coordinate difference between adjacent coordinate points in the cursor movement trajectory;

[0265] According to the coordinate difference, it is determined whether there is a mutation point in the cursor movement trajectory.

[0266] Furthermore, when the element selection program based on the move operation is executed by the processor, the following operations are also implemented:

[0267] Arrange the coordinate differences in descending order to obtain a sorted list;

[0268] Obtaining the median of the sorted list;

[0269] Calculate the median difference between each coordinate difference and the median;

[0270] Determining whether the median difference exceeds a preset difference threshold;

[0271] If the median difference exceeds a preset difference threshold, the coordinate point corresponding to the median difference is identified as a mutation point in the cursor movement trajectory.

[0272] In this embodiment, when an element selection instruction triggered by the user in the current interface is received, the cursor movement operation input by the user based on the current interface is detected, the corresponding cursor movement trajectory is determined according to the cursor movement operation, and a closed area is generated based on the cursor movement trajectory. The cursor can be moved at will as needed, so as to adapt to complex relationship diagram arrangements and select any element; it is determined whether there are interface elements in the closed area in the current interface. If there are interface elements in the closed area in the current interface, the interface elements in the closed area are selected. The closed area moved out according to the required operation can adapt to the concentrated and complex arrangement of the relationship diagram, thereby circling the required interface elements and improving the accuracy of interface element selection.

[0273] In addition, refer to Figure 8 The embodiment of the present invention further provides an element selection device based on a moving operation, and the element selection device based on a moving operation includes:

[0274] The detection module 10 is used to detect the cursor movement operation input by the user based on the current interface when receiving the element selection instruction triggered by the user in the current interface.

[0275] The determination module 20 is used to determine the corresponding cursor movement track according to the cursor movement operation.

[0276] The generating module 30 is used to generate a closed area based on the cursor movement trajectory.

[0277] The judgment module 40 is used to judge whether there is an interface element in the current interface that is within the closed area.

[0278] The selection module 50 is used to select the interface element in the closed area if there is an interface element in the current interface in the closed area.

[0279] In this embodiment, when an element selection instruction triggered by the user in the current interface is received, the cursor movement operation input by the user based on the current interface is detected, the corresponding cursor movement trajectory is determined according to the cursor movement operation, and a closed area is generated based on the cursor movement trajectory. The cursor can be moved at will as needed, so as to adapt to complex relationship diagram arrangements and select any element; it is determined whether there are interface elements in the closed area in the current interface. If there are interface elements in the closed area in the current interface, the interface elements in the closed area are selected. The closed area moved out according to the required operation can adapt to the concentrated and complex arrangement of the relationship diagram, thereby circling the required interface elements and improving the accuracy of interface element selection.

[0280] Other embodiments or specific implementations of the element selection device based on movement operation described in the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0281] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for selecting elements based on a moving operation, characterized in that: The element selection method based on the move operation comprises the following steps: When receiving an element selection instruction triggered by the user in the current interface, detecting a cursor movement operation input by the user based on the current interface; Determine a corresponding cursor movement trajectory according to the cursor movement operation; Generate a closed area based on the cursor movement trajectory; Determine whether there is an interface element in the current interface that is within the closed area; If there is an interface element in the closed area in the current interface, then select the interface element in the closed area; After the step of determining whether there is an interface element in the current interface that is within the closed area, the method further includes: Determine whether there is an interface element to be confirmed whose area ratio is less than or equal to a first preset ratio and greater than a second preset ratio, the second preset ratio is less than the first preset ratio, the area ratio is the area ratio of the interface element to be confirmed in the closed area, and the interface element to be confirmed is the interface element traversed in the current interface; If so, the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio is identified as the element to be identified; Performing text recognition on the element to be recognized to obtain text information to be recognized; Performing text recognition on the interface elements within the closed area to obtain target text information; It is determined whether the element to be identified is within the closed area according to the text information to be identified and the target text information.

2. The element selection method based on movement operation according to claim 1, characterized in that: The determining whether there is an interface element in the current interface that is within the closed area specifically includes: Traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements; Get the center point of the current interface element; Generate a target ray along a preset direction with the center point as the starting point; Obtaining the number of intersections between the target ray and the edge of the closed area; It is determined whether the current interface element is within the closed area according to the number of intersections.

3. The element selection method based on movement operation as claimed in claim 2, characterized in that: The determining, according to the number of intersections, whether the current interface element is within the closed area specifically includes: Determine whether the number of intersection points is an odd number; If the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

4. The element selection method based on movement operation according to claim 1, characterized in that: The determining whether there is an interface element in the current interface that is within the closed area specifically includes: Traversing the interface elements in the current interface, and taking the traversed interface elements as the interface elements to be confirmed; Obtaining the element area of ​​the interface element to be confirmed; Calculating the area ratio of the interface element to be confirmed within the closed area according to the element area; It is determined whether the interface element to be confirmed is within the closed area according to the area ratio.

5. The element selection method based on movement operation as claimed in claim 4, characterized in that: The determining whether the to-be-confirmed interface element is within the closed area according to the area ratio specifically includes: Determining whether the area ratio is greater than a first preset ratio; If the area ratio is greater than the first preset ratio, it is determined that the interface element to be confirmed is within the closed area.

6. The element selection method based on movement operation according to claim 1, characterized in that: The determining, according to the to-be-recognized text information and the target text information, whether the to-be-recognized element is within the closed area specifically includes: Calculating the text similarity between the to-be-recognized text information and the target text information; It is determined whether the element to be identified is within the closed area according to the text similarity.

7. The element selection method based on movement operation as claimed in claim 6, characterized in that: The calculating of the text similarity between the to-be-recognized text information and the target text information specifically includes: Extracting keywords from the text information to be identified to obtain keywords of the text to be identified; Extracting keywords from the target text information to obtain keywords of the target text; The text similarity between the text information to be recognized and the target text information is calculated according to the keywords of the text to be recognized and the keywords of the target text.

8. The element selection method based on movement operation according to claim 1, characterized in that: There are multiple interface elements in the closed area; The determining, according to the to-be-recognized text information and the target text information, whether the to-be-recognized element is within the closed area specifically includes: Clustering target text information of multiple interface elements within the closed area to obtain a cluster center; Calculating the clustering similarity between the text information to be identified and the cluster center; According to the cluster similarity, it is determined whether the element to be identified is within the closed area.

9. The element selection method based on movement operation as claimed in claim 8, characterized in that: The step of judging whether the element to be identified is within the closed area according to the cluster similarity specifically includes: Determining whether the cluster similarity exceeds a preset similarity threshold; When the cluster similarity exceeds the preset similarity threshold, it is determined that the element to be identified is within the closed area.

10. The element selection method based on movement operation according to any one of claims 1 to 9, characterized in that: The generating of a closed area based on the cursor movement trajectory specifically includes: Determine whether there is a mutation point in the cursor movement trajectory; If there is a mutation point in the cursor movement trajectory, taking the coordinate points in the cursor movement trajectory other than the mutation point as the target coordinate point; Connect the target coordinate points to obtain a closed area.

11. The element selection method based on movement operation according to claim 10, characterized in that: The determining whether there is a mutation point in the cursor movement trajectory specifically includes: Calculating the coordinate difference between adjacent coordinate points in the cursor movement trajectory; According to the coordinate difference, it is determined whether there is a mutation point in the cursor movement trajectory.

12. The element selection method based on movement operation according to claim 11, characterized in that: The determining, according to the coordinate difference, whether there is a mutation point in the cursor movement trajectory specifically includes: Arrange the coordinate differences in descending order to obtain a sorted list; Obtaining the median of the sorted list; Calculate the median difference between each coordinate difference and the median; Determining whether the median difference exceeds a preset difference threshold; If the median difference exceeds a preset difference threshold, the coordinate point corresponding to the median difference is identified as a mutation point in the cursor movement trajectory.

13. An element selection device based on mobile operation, characterized in that: The element selection device based on moving operation includes: a memory, a processor, and an element selection program based on moving operation stored in the memory and executable on the processor. When the element selection program based on moving operation is executed by the processor, the steps of the element selection method based on moving operation as described in any one of claims 1 to 12 are implemented.

14. A storage medium, characterized in that: The storage medium stores an element selection program based on a move operation, and when the element selection program based on a move operation is executed by a processor, the steps of the element selection method based on a move operation as described in any one of claims 1 to 12 are implemented.

15. An element selection device based on movement operation, characterized in that: The element selection device based on the moving operation comprises: A detection module, configured to detect a cursor movement operation input by a user based on the current interface when receiving an element selection instruction triggered by the user in the current interface; A determination module, used to determine a corresponding cursor movement trajectory according to the cursor movement operation; A generating module, used for generating a closed area based on the cursor movement trajectory; A judging module, used for judging whether there is an interface element in the current interface that is within the closed area; A selection module, configured to select an interface element within the closed area if there is an interface element within the closed area in the current interface; The judgment module is also used to judge whether there is an interface element to be confirmed whose area ratio is less than or equal to a first preset ratio and greater than a second preset ratio, the second preset ratio is less than the first preset ratio, the area ratio is the area ratio of the interface element to be confirmed in the closed area, and the interface element to be confirmed is the interface element traversed in the current interface; if it exists, the interface element to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio is identified as the element to be identified; text recognition is performed on the element to be identified to obtain text information to be identified; text recognition is performed on the interface element in the closed area to obtain target text information; and based on the text information to be identified and the target text information, it is judged whether the element to be identified is in the closed area.

16. The element selection device based on movement operation as claimed in claim 15, characterized in that: The judgment module is further used to traverse the interface elements in the current interface, and use the traversed interface elements as the current interface elements; and obtain the center point of the current interface element; Generate a target ray along a preset direction with the center point as the starting point; Obtain the number of intersections between the target ray and the edge of the closed area; and determine whether the current interface element is within the closed area based on the number of intersections.

17. The element selection device based on movement operation as claimed in claim 16, characterized in that: The judgment module is further used to judge whether the number of intersections is an odd number; if the number of intersections is an odd number, it is determined that the current interface element is within the closed area.

18. The element selection device based on movement operation as claimed in claim 15, characterized in that: The judgment module is further used to traverse the interface elements in the current interface, and use the traversed interface elements as the interface elements to be confirmed; and obtain the element area of ​​the interface elements to be confirmed; According to the element area, the area ratio of the interface element to be confirmed within the closed area is calculated; and according to the area ratio, whether the interface element to be confirmed is within the closed area is determined.

19. The element selection device based on movement operation as claimed in claim 15, characterized in that: The generation module is also used to determine whether there is a mutation point in the cursor movement trajectory; if there is a mutation point in the cursor movement trajectory, the coordinate points in the cursor movement trajectory other than the mutation point are used as target coordinate points; and the target coordinate points are connected to obtain a closed area.

Images