A patent document drawing display method, device and terminal device

Abstract

The present application provides a method, device, equipment and storable medium for displaying drawings of patent documents, which are applicable to the field of computer technology. The method includes: receiving a request from a user for displaying drawings of patent documents, wherein the request includes search text information and a set of drawing numbers, wherein the set of drawing numbers consists of at least two drawing numbers; determining the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings of the patent document and the set of drawing numbers; extracting semantic features of the initial drawings to be displayed and the search text information based on multiple preset scale feature windows, generating alignment features of drawings to be displayed and search text alignment features; matching and analyzing the alignment features of drawings to be displayed with the search text alignment features, determining the target drawings to be displayed, and displaying them to the user. The present application is used to display patent drawings that meet the search content, and help users quickly understand the technical solution.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a method, device and terminal device for displaying drawings in patent documents. Background Art

[0002] Since the implementation of the patent system, the number of patent applications in my country has been increasing year by year, and major patent websites have also taken advantage of this opportunity to provide patent downloads and online patent reading services. However, since the specification and the drawings are separate in patent documents, it is necessary to compare them before and after when reading to understand the relationship between the various figure marks in the drawings, so it takes a lot of effort to understand the solutions recorded in the patent documents.

[0003] In order to facilitate readers' reading, the patent website will provide two display areas. The first display area is the text display area, which is used to display the text part of the patent document, such as the claims and the description; the second display area is the figure display area, which is used to display the drawings of the description, so that readers can view the patent document text and the patent document drawings at the same time, which is convenient for matching the figure numbers of the text part with the figure numbers corresponding to the figure part.

[0004] However, when readers read such patent documents through the above method, they need to first find the partial enlarged diagram containing the current content according to the current content. When there are many drawings in the patent document and the drawing numbers are many, it takes a lot of time and effort to find the partial enlarged diagram containing the current content in the drawings. Therefore, the display of the drawings in the patent document is not intuitive enough. Therefore, it is necessary to improve it. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, apparatus and terminal device for displaying drawings in a patent document, which can solve the problem in the prior art that the display of drawings in patent documents is not intuitive enough.

[0006] A first aspect of an embodiment of the present application provides a method for displaying drawings of a patent document, comprising:

[0007] Receive a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers;

[0008] Determine the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set;

[0009] Based on a plurality of preset scale feature windows, semantic features are extracted from the initial drawings to be displayed and the search text information respectively to generate alignment features of the drawings to be displayed and alignment features of the search text;

[0010] The alignment features of the to-be-displayed figures are matched and analyzed with the alignment features of the searched texts to determine the target to-be-displayed figures for display to the user.

[0011] A second aspect of the embodiments of the present application provides a device for displaying drawings of patent documents, including:

[0012] A patent document drawing display request receiving module, used to receive a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers;

[0013] An initial drawing to be displayed determining module, used for determining the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set;

[0014] An alignment feature generation module, configured to extract semantic features of the initial drawings to be displayed and the search text information based on a plurality of preset scale feature windows, and generate alignment features of the drawings to be displayed and alignment features of the search text;

[0015] The target drawings to be displayed are determined, and are used to match and analyze the alignment features of the drawings to be displayed with the alignment features of the search text, so as to determine the target drawings to be displayed for display to the user.

[0016] The third aspect of an embodiment of the present application provides a terminal device, which includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the steps of the patent document illustration display method as described in any one of the above-mentioned first aspects.

[0017] The fourth aspect of an embodiment of the present application provides a computer-readable storage medium, comprising: storing a computer program, characterized in that when the computer program is executed by a processor, the steps of the method for displaying drawings of a patent document as described in any one of the first aspects above are implemented.

[0018] Compared with the prior art, the beneficial effect of the embodiment of the present application is as follows: by extracting the figure numbers from the text of the patent document that the user is currently reading and putting them into a figure number set, the patent document figures are screened once according to the figure number set, the screened picture set is screened again according to the search text, and the patent document figures obtained by the secondary screening are displayed, so that the user can quickly see the figures that are highly related to the current reading content, which can help the reader to more quickly understand the solution proposed by the current patent document. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 It is a schematic diagram of the implementation flow of the method for displaying drawings of patent documents provided in Example 1 of the present application;

[0021] Figure 2 It is a schematic diagram of the implementation process of the method for displaying drawings of patent documents provided in Example 2 of this application;

[0022] Figure 3 It is a schematic diagram of the implementation process of the method for displaying drawings of patent documents provided in Example 3 of this application;

[0023] Figure 4 It is a schematic diagram of the implementation flow of the method for displaying drawings of patent documents provided in Example 4 of the present application;

[0024] Figure 5 It is a schematic diagram of the implementation process of the method for displaying drawings of patent documents provided in Example 5 of the present application;

[0025] Figure 6 It is a schematic diagram of the implementation flow of the method for displaying drawings of patent documents provided in Example 6 of the present application;

[0026] Figure 7 It is a schematic diagram of the implementation process of the patent document drawing display method provided in Example 7 of this application;

[0027] Figure 8 It is a schematic diagram of the structure of the device for displaying the drawings of the patent document provided in the embodiment of the present application;

[0028] Fig. 9 It is a schematic diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0030] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0031] Figure 1 The following is a flowchart of the method for displaying drawings of patent documents provided in the first embodiment of the present application, which is described in detail as follows:

[0032] Step S101, receiving a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers.

[0033] In this embodiment, the set of figure numbers contained in the received request is composed of the figure numbers extracted from the content currently read by the user and the user's search text for the patent document, so as to determine which parts are included in the part currently read by the user, so as to facilitate the screening of the patent document figures according to the set of figure numbers and the search content, so as to obtain the patent document figures with a high degree of relevance to the content currently read by the user. Secondly, by screening the patent document figures with a set of figure numbers composed of multiple different figure numbers, it is possible to filter out the figures that are related to the content currently read by the user but have a low degree of relevance, thereby ensuring that the obtained patent document figures have a high degree of relevance.

[0034] Step S102, determining the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set.

[0035] In this embodiment, the drawings of the patent document are screened according to the mutual inclusion relationship between the drawing numbers of the drawings of the patent document and the drawing number set, so as to ensure that the drawings to be displayed are highly relevant to the content currently read by the user. When the user opens the patent document for the first time, the drawing numbers of the drawings of the patent document will automatically extract all the drawing numbers from the drawings of the patent document and put them into the drawing number set of the drawings of the patent document corresponding to the drawings of the patent document from which the drawing numbers are extracted. The drawing number set of the patent document will determine the initial drawings to be displayed according to the mutual inclusion relationship with the drawing number set reflecting the content currently read by the user, and will be used for subsequent secondary screening according to the user's search content of the patent document.

[0036] Step S103 : Based on a plurality of preset scale feature windows, semantic features are extracted from the initial drawings to be displayed and the search text information respectively, and alignment features of drawings to be displayed and alignment features of search text are generated.

[0037] In this embodiment, the scale feature window refers to the parameter regarded as the scale introduced in the network architecture, and the scale space representation sequence under multiple scales is obtained by continuously changing the scale parameter, which is used to extract the main contour of the scale space of these sequences. Since there are huge differences in the expression form, semantics, information density, etc. between the two data of the figure and the text, there will be a problem of information misalignment between the data. Therefore, the figure feature vector and the retrieval text vector extracted from the figure information and the retrieval text information need to be extracted through different scale feature windows, and then the extracted figure features of the initial figure to be displayed and the text features of the retrieval text information are processed by feature alignment. By mapping the information in the two forms of figure and text into the same high-dimensional space, an alignment scaling vector is dynamically generated, so that the feature representation of the two forms of data is semantically consistent through the feature alignment process. The alignment scaling vector generated by the figure features of the initial figure to be displayed is the alignment feature of the figure to be displayed, and the alignment scaling vector generated by the text features of the retrieval text information is the retrieval text alignment feature.

[0038] Step S104, matching and analyzing the alignment features of the to-be-displayed drawings with the alignment features of the searched texts to determine the target to-be-displayed drawings for display to the user.

[0039] In this embodiment, the illustration alignment features can be directly compared with the search text alignment features, so that the target illustration to be displayed can be determined from the initial illustrations to be displayed based on the content of the search text. Alternatively, the illustration alignment features and the search text alignment features can be further mapped to a higher-dimensional space for comparison to improve the matching accuracy, so that the target illustration to be displayed can be determined from the initial illustrations to be displayed based on the comparison results in the higher-dimensional space.

[0040] The method for displaying drawings in patent documents provided in the embodiment of the present application extracts drawing numbers from the text of the patent document that the user is currently reading and puts them into a drawing number set, screens the drawings in the patent document once according to the drawing number set, screens the screened picture set again according to the search text, and displays the drawings in the patent document obtained by the second screening, so that the user can quickly see the drawings that are highly related to the current reading content, which can help the reader more quickly understand the solution proposed in the current patent document.

[0041] Figure 2 The flowchart of the method for displaying drawings of patent documents provided in the second embodiment of the present application is shown. The difference between the method and the first embodiment is that the step S102 specifically includes:

[0042] Step S201, based on the mutual inclusion relationship between the figure numbers of the figures in the patent document and the figure number set, determine the figures in the patent document that contain the figure number set and have the least total number of figure numbers.

[0043] In this embodiment, there may be two situations for the patent document drawings containing the set of drawing numbers and the least total number of drawing numbers. The first situation is that the mechanism corresponding to the set of drawing numbers, that is, the mechanism corresponding to the content currently read by the user is only a part of the drawing, and, compared with other drawings containing the mechanism corresponding to the content currently read by the user, the drawing contains the least parts of the mechanism that is not currently read by the user, that is, the drawing that shows the mechanism corresponding to the content currently read by the user most clearly. The second situation is that the set of drawing numbers is equal to the set composed of all the drawing numbers in the drawing, that is, the drawing is essentially a partial enlarged view of the mechanism corresponding to the content currently read by the user, and, in this case, since the drawing does not contain parts of other mechanisms, there are drawings of the first situation and drawings of the second situation in the drawings of the patent document at the same time, that is, when the mechanism corresponding to the content currently read by the user is only a part of the drawing and the partial enlarged view of the mechanism corresponding to the content currently read by the user coexist, only the drawing of the second situation will be selected, so that the selected drawing is the most clear drawing showing the mechanism corresponding to the content currently read by the user, thereby facilitating the user to understand the mechanism corresponding to the content currently read.

[0044] Step S202, based on the set of figure numbers, extracting the initial figures to be displayed from the figures of the patent document that contain the set of figure numbers and have the least total number of figure numbers.

[0045] In this embodiment, there may be two situations for the patent document drawings that contain the set of drawing numbers and have the least total number of drawing numbers. The first situation is that the mechanism corresponding to the content currently read by the user is only a part of the drawing, and the second situation is that the drawing is a partial enlarged view of the mechanism corresponding to the content currently read by the user. In the first situation, since the drawing also includes components of other mechanisms, it may interfere with the reader's finding the mechanism corresponding to the currently read content. Therefore, cutting out the patent document drawings that only contain the mechanism corresponding to the content currently read by the user as the patent document drawings to be displayed can exclude the influence of the components of other mechanisms and facilitate user understanding. In the second situation, cutting out the partial enlarged view of the mechanism corresponding to the content currently read by the user can remove the white background part outside the drawing, reduce the area occupied by the drawing, and facilitate its subsequent display.

[0046] In this embodiment, preferably, the pointing positions of each indicator line corresponding to each figure number in the figure number set are determined in the figure drawing of the patent document that contains the figure number set and has the least total number of figure numbers; the figure content corresponding to each figure number in the figure number set is determined based on the indicator position of each indicator line; all figure numbers and corresponding indicator lines are deleted in the figure drawing of the patent document that contains the figure number set and has the least total number of figure numbers; the figures to be displayed are cut out in the figure drawing of the patent document that contains the figure number set and has the least total number of figure numbers based on the figure content corresponding to each figure number; new indicator lines and corresponding figure numbers are added in the figure to be displayed based on the figure content corresponding to each figure number.

[0047] The patent document illustration display method provided in the embodiment of the present application determines the content of the illustration corresponding to each figure number through the pointing position of each figure number and the corresponding indicator line in the figure number set, and these figure contents are the specific structural composition of the mechanism corresponding to the user's current reading, that is, the part that needs to be displayed to the user. Then, after recording each figure number and the corresponding figure content in the figure number set, all the figure numbers and corresponding indicator lines in the patent document figure that contains the figure and has the least total number of figure numbers are deleted to avoid the figure numbers and indicator lines on the figure being partially intercepted, so that the user mistakenly believes that the partially intercepted figure numbers and indicator lines are part of the structure displayed in the patent document figure. After intercepting the figure to be displayed, the figure numbers and corresponding indicator lines are supplemented according to the corresponding relationship between the figure numbers and the figure contents, so as to facilitate the user to find the corresponding parts and understand the specific structural composition and corresponding connection relationship of the mechanism corresponding to the current reading content.

[0048] Figure 3 The flowchart of the method for displaying drawings of patent documents provided in the third embodiment of the present application is shown. The difference between the method and the second embodiment is that the step S202 specifically includes:

[0049] Step S301, determining the position of each figure number in the figure number set and the position of each indicator line corresponding to the figure number in the figure number set of the patent document drawing that contains the figure number set and has the least total number of figure numbers.

[0050] In this embodiment, since the position of the figure numbers in the figure number set cannot be predicted in the figure, when the figure numbers in the figure number set are all located on one side of the figure, the figure needs to be processed accordingly before it can be intercepted. In this embodiment, the rectangular coordinate system will be used as an example to illustrate the processing of the figure in the above scenario: determine the figure number in the figure number set and the position of the corresponding indicator line in the figure number set to determine the object to be operated.

[0051] Step S302: determining the endpoint coordinates of each indicator line away from the drawing number according to the position of each drawing number and the position of each indicator line corresponding to the drawing number.

[0052] In this embodiment, the coordinates of the endpoint of the indicator line away from the end of the drawing number are the coordinates of the pixel point to which the indicator line points.

[0053] Step S303, determining the center coordinates of the drawing according to the coordinates of the endpoints of the respective indicator lines away from the drawing number.

[0054] In this embodiment, the method for determining the center coordinates of the drawing according to the endpoint coordinates is: respectively calculate the mean of the endpoint coordinates based on the x-axis and the y-axis, and the obtained coordinates are the center coordinates of the drawing. For example: there are 10 drawing numbers in a drawing of a patent document, and the intersection of the number set composed of the above 10 drawing numbers and the drawing number set is only 6, and the position pointed by the indicator line of these 6 drawing numbers, that is, the endpoint coordinates of the indicator line away from one end of the drawing number are: (1, 1), (9, 1), (6, 2), (2, 4), (1, 5), (9, 5), and the center coordinates of the drawing calculated according to these 6 endpoint coordinates are: (3, 3).

[0055] Step S304, based on the center coordinates of the drawing, the endpoint coordinates of each indicator line away from the drawing number and the preset indicator line length, determine the positions of each target indicator line corresponding to the drawing number in the patent document drawing that contains the drawing number set and has the least total number of drawing numbers, so that the extension lines of each target indicator line intersect with the center coordinates of the drawing.

[0056] In this embodiment, the preset indicator line length can be a static setting, such as a length of 10 pixels, 5 mm, etc., or a dynamic setting, such as a quarter or a sixth of the shorter side of the length and height of the patent document drawing containing the drawing number set and the least total drawing number, etc., and the present application does not impose specific restrictions on the preset length. After recording the endpoint coordinates of each drawing number in the drawing number set and the corresponding each indicator line away from one end of the drawing number, all the drawing numbers and corresponding indicator lines in the patent document drawing containing the drawing number set and the least total drawing number are deleted to avoid intercepting part of the drawing number or indicator line during interception, so that the user mistakenly believes that the partially intercepted drawing number or indicator line is part of the structure shown in the drawing. Then, the target indicator line is drawn according to the endpoint coordinates and the center coordinates of the drawing, so that the extension lines of the target indicator line pass through the center coordinates of the drawing, so that the target indicator line diverges to the surroundings based on the center coordinates of the drawing.

[0057] Step S305: determining the position of the target figure number according to the positions of the target indication lines corresponding to the figure number.

[0058] In this embodiment, according to each figure number and the endpoint coordinates of each corresponding indicator line away from the figure number, the corresponding target figure number is added at the end of the target indicator line away from the endpoint coordinates, so that the user can recognize the corresponding relationship between the figure content and the figure number. If the target figure number overlaps with the figure structure in the figure content when adding the target figure number, the length of the target indicator line corresponding to the target figure number can be appropriately extended / shortened to avoid the overlap between the target figure number and the figure structure in the figure content. Whether to extend the target indicator line or shorten the target indicator line is determined by the shorter of the extended length and the shortened length, and the shortened length must be less than the preset indicator line length.

[0059] Step S306, based on the position of the target figure number, extract the initial figure to be displayed from the figures of the patent document that contain the figure number set and have the least total number of figure numbers.

[0060] In this embodiment, after the above processing, the target figure number will radiate from the center coordinates of the figure in the patent document figure that contains the figure number set and has the least total number of figure numbers, surrounding the four sides of the mechanism corresponding to the figure number set. Then, the positions of the four interception edges are determined according to the positions of all the target figure numbers, so that all the target figure numbers can be located in the rectangle formed by the four interception edges, and the area of ​​the rectangle is the smallest. The patent document figure that contains the figure number set and has the least total number of figure numbers is intercepted according to the four interception edges, and the intercepted figure is determined as the initial figure to be displayed, which is used for subsequent secondary screening according to the search text to ensure that the patent images displayed meet the needs of users.

[0061] The patent document illustration display method provided in the embodiment of the present application, after the above processing, makes the target illustration number diverge around the center coordinate of the illustration, so that it is possible to select the patent document illustration containing the said illustration number set and the least total number of illustration numbers based on the target illustration number, and ensure that the intercepted illustration includes the mechanism corresponding to the content currently read by the user. In addition, even if parts of other mechanisms may be intercepted during the interception process, there is no figure number pointing to the part and the corresponding indicator line in the intercepted illustration to be displayed, so it will not interfere with the user's understanding of the solution.

[0062] Figure 4 The flowchart of the method for displaying drawings of patent documents provided in the fourth embodiment of the present application is shown. The difference between the fourth embodiment and the first embodiment is that the step S104 specifically includes:

[0063] Step S401 : performing a multi-scale convolution operation on a plurality of preset scale feature windows and the initial figure to be displayed to obtain a plurality of local representation feature maps of the figure to be displayed.

[0064] In this embodiment, feature extraction is performed on the local representation feature map of the figure to be displayed through multiple scale feature windows, so as to effectively extract the figure information and avoid the problem of feature alignment failure in the future, so as to improve the effectiveness of the subsequent matching with the search text. Among them, the feature window is a matrix for convolution processing of multidimensional data in the multi-scale convolution operation, which is used to extract feature information from the input multi-dimensional data. Feature windows of different sizes realize the multi-scale convolution operation on the data by moving on the pixel data of the initial figure to be displayed and multiplying the data with the corresponding value in the window to obtain and output the local representation feature map of the figure to be displayed. The shape of the feature window depends on the height and width of the convolution kernel, and the size and shape determine the area of ​​concern for each convolution operation. By performing multi-scale convolution operations on the data through feature windows, the computing network is helped to learn the local features of multi-dimensional data and improve the understanding and processing capabilities of the input data.

[0065] Step S402: performing a multi-scale convolution operation on a plurality of preset scale feature windows and the search text information to obtain a plurality of search text local representation feature maps.

[0066] In this embodiment, a multi-scale convolution operation is performed on the retrieval text information through multiple scale feature windows. This is a process in which feature windows of different sizes are moved on the binary format data corresponding to the retrieval text information and the data is multiplied by the corresponding value in the window to obtain and output a local representation feature map of the retrieval text, thereby obtaining local representation feature maps of the retrieval text at multiple scales, thereby realizing the extraction of the retrieval text information through multiple scale feature windows, which is used to effectively extract text information, avoid the subsequent problem of feature alignment failure, and improve the reliability of subsequent matching with the initial illustration information to be displayed.

[0067] Step S403, calculating the average value of all values ​​in the local representation feature graphs of the drawings to be displayed, to obtain a plurality of global representation feature graphs of the drawings to be displayed.

[0068] In this embodiment, the average value of the local representation feature map of the figure to be displayed at each scale is calculated, that is, the average value of all values ​​in the local representation feature map of a single figure to be displayed is calculated, so as to obtain a one-dimensional vector with the same number of scales, which is used to reduce the size of the feature map, reduce the amount of calculation and memory consumption. All the extracted one-dimensional vectors at different scales are spliced ​​to generate a vector map as the global representation feature of the figure to be displayed, that is, the global representation feature map of the figure to be displayed.

[0069] Step S404, calculating the average value of all values ​​in each of the local representation feature graphs of the search text to obtain a plurality of global representation feature graphs of the search text.

[0070] In this embodiment, by calculating the average value of the local representation feature map of the search text at each scale, that is, calculating the average value of all values ​​in the local representation feature map of a single search text, a one-dimensional vector with the same number of scales is obtained to reduce the size of the global representation feature map, reduce the amount of calculation and memory consumption. All the one-dimensional vectors extracted at different scales are spliced ​​to generate a vector map as the global representation feature of the search text, that is, the global representation feature map of the search text.

[0071] Step S405 , performing aggregation operations on the global representation feature graph of the to-be-displayed drawing and the global representation feature graph of the search text, respectively, to generate alignment features of the to-be-displayed drawing and alignment features of the search text.

[0072] In this embodiment, the global representation feature map of the figure to be displayed and the global representation feature map of the search text are aligned to realize the feature alignment operation between the figure and the text. The feature alignment operation can be to aggregate the feature maps of multiple scales by direct addition, or to aggregate the feature maps of multiple scales by weighted summation, wherein the weights are used to highlight certain important features, or to average the feature maps of multiple scales, or to find the variance of the feature maps of multiple scales. The aggregation result is the alignment feature of the figure. The vector graph obtained by the aggregation calculation of the global representation feature map of the figure to be displayed is the alignment feature of the figure to be displayed, and the vector graph obtained by the aggregation calculation of the global representation feature map of the search text is the alignment feature of the search text.

[0073] The patent document illustration display method provided in the embodiment of the present application captures the association relationship between the initial illustration to be displayed and the search text through multiple scale feature windows, thereby obtaining rich semantic information in the features, and uses the rich semantic information as the feature alignment basis of the initial illustration content to be displayed and the search text content, so as to improve the feature alignment accuracy between the initial illustration to be displayed and the search text; generates a global representation feature map by extracting the average value in the feature map, thereby retaining more information in the local feature map, compressing the feature map size, and reducing the amount of calculation and memory consumption; and realizes mutual matching and retrieval of the initial illustration content to be displayed and the search text content by aligning and calculating the global representation feature map.

[0074] Figure 5 The flowchart of the method for displaying drawings of patent documents provided in the fifth embodiment of the present application is shown. The difference between the fifth embodiment and the fourth embodiment is that the step S405 specifically includes:

[0075] Step S501 , performing matrix operations on the global representation feature maps of the plurality of drawings to be displayed according to a preset image unit transformation matrix, a preset image-text matching parameter matrix and a preset image-text feature mapping matrix, to obtain alignment vectors of the drawings to be displayed.

[0076] In this embodiment, the image unit transformation matrix can be a fixed parameter set by default inside the module, which is used to transform the dimension of the feature information matrix; the image-text matching parameter matrix and the image-text feature mapping matrix are the default learnable parameters inside the module. During the training process of the computing model, the two parameter matrices, the image-text matching parameter matrix and the image-text feature mapping matrix, are continuously updated and iterated; the image unit transformation matrix and the image-text matching parameter matrix can be first multiplied with the global representation feature map of the illustration to be displayed, and then the multiplication result can be transposed, and then the transposed result can be multiplied with the image-text feature mapping matrix, and the calculation result is the alignment vector of the illustration to be displayed at a single scale.

[0077] Step S502: performing weighted summation on the alignment vectors of the drawings to be displayed to obtain alignment features of the drawings to be displayed.

[0078] In this embodiment, a weighted sum is calculated for the alignment vectors of the to-be-displayed figures at all different scales, and the features at different scales are aggregated. The aggregation result is the alignment feature of the to-be-displayed figures, which is used for matching with the search text.

[0079] Step S503, performing matrix operations on the plurality of retrieval text global representation feature maps according to a preset text unit transformation matrix, a preset image-text matching parameter matrix and a preset image-text feature mapping matrix, to obtain a retrieval text alignment vector.

[0080] In this embodiment, the image unit transformation matrix and the image-text matching parameter matrix are first multiplied with the global representation feature map of the retrieved text, and then the multiplication result is transposed. The transposed result is then multiplied with the image-text feature mapping matrix, and the calculated result is the retrieval text alignment vector at a single scale.

[0081] Step S504: performing weighted summation on the search text alignment vectors to obtain search text alignment features.

[0082] In this embodiment, a weighted sum is calculated for the search text alignment vectors at all different scales to aggregate the features at different scales. The aggregation result is the search text alignment feature, which is used to perform a matching operation with the initial figure to be displayed.

[0083] The patent document illustration display method provided in the embodiment of the present application scales and aligns the global representation feature map of the illustration to be displayed and the global representation feature map of the search text, thereby aligning the features of the illustration to be displayed and the search text of different scales on the same dimension, and aggregating the scaled and aligned features of the illustration to be displayed and the search text features, thereby effectively reducing the dimension of the global feature expression, preventing overfitting in the calculation process, and improving the robustness of the feature expression and calculation analysis process.

[0084] The patent document drawing display method provided in the embodiment of the present application is:

[0085] Figure 6 The flowchart of the method for displaying drawings of patent documents provided in the sixth embodiment of the present application is shown. The difference between the sixth embodiment and the first embodiment is that the step S104 specifically includes:

[0086] Step S601, calculating the Euclidean distance between the alignment feature of the to-be-displayed figure and the alignment feature of the search text.

[0087] In this embodiment, for the alignment features of the drawings to be displayed and the alignment features of the search text, the Euclidean distance is used as the retrieval metric for searching from the text to the drawings to be displayed. The Euclidean distance refers to the real distance between two points in a multidimensional space, and in this application, it refers to the straight-line distance between the alignment features of the drawings to be displayed and the alignment features of the search text in a high-dimensional space.

[0088] Step S602: Calculate the matching probability between the alignment feature of the to-be-displayed drawing and the alignment feature of the search text according to the Euclidean distance.

[0089] In this embodiment, the Euclidean distance may be normalized by calculating the mean or the standard deviation, and then the normalized result may be subjected to a balanced analysis. The balanced analysis may be a continuous histogram equalization method or a discrete histogram equalization method. The one-to-one matching probability between the alignment features of the to-be-displayed drawings and the alignment features of the searched text may be obtained through the balanced analysis.

[0090] Step S603: when the matching probability is greater than a preset matching probability threshold, the initial drawing to be displayed corresponding to the alignment feature of the drawing to be displayed is determined as the target drawing to be displayed.

[0091] In this embodiment, the preset matching probability threshold can be set manually, which can be 70% or 80%. When the matching probability is greater than the preset matching probability threshold, it means that the content in the initial figure to be displayed is consistent with the content in the search text, and the initial figure to be displayed is determined as the target figure to be displayed for subsequent display to the user; when the matching probability is less than the preset matching probability threshold, it means that the content in the initial figure to be displayed is inconsistent with the content in the search text, and the initial figure to be displayed is not highly correlated with the search text, and it is necessary to continue to calculate the matching probability for re-matching.

[0092] The patent document illustration display method provided in the embodiment of the present application uses Euclidean distance as a matching metric, thereby reducing the computational complexity of matching the initial illustration to be displayed with the search text. By calculating the matching probability between the initial illustration to be displayed and the search text, the initial illustration to be displayed is screened again, and the patent illustration that best meets the user's needs is further screened out from the initial illustration to be displayed through the search text for display, thereby saving the user's time and energy in screening the patent illustrations and facilitating the user to quickly understand the technical solution in the patent.

[0093] Figure 7 The flowchart of the method for displaying drawings of patent documents provided in the seventh embodiment of the present application is shown. The difference between the method and the first embodiment is that after the step S104, the method further includes:

[0094] Step S701: receiving a highlighting / cancelling highlighting instruction of the content of the drawing sent by the user based on the drawing to be displayed for the purpose.

[0095] In this embodiment, since the correspondence between the drawing number and the drawing content in the patent document drawing highlighted to the user may be wrong, when the above situation occurs, the user can manually adjust the drawing content of the patent document for highlighting, highlight certain structures in the drawing or cancel the highlighting of certain structures. This application does not limit how the user performs the highlighting / cancelling operation, which can be left-clicking the drawing content in the displayed patent document drawing, or selecting the drawing content of the patent document and clicking the right-click menu, or selecting the drawing content of the patent document and clicking a preset designated button in the drawing display area.

[0096] When the user performs the highlight / cancel highlight operation, it involves the selection of the content of the patent document drawings to be displayed. The selection of the content of the patent document drawings has the following situations: the first situation is: the user selects a line segment in the patent document drawings to be displayed, in which case the line segment is regarded as the content selected by the user; the second situation is: the user selects a background pixel point in the patent document drawings to be displayed, in which case the closed area containing the background pixel point and the smallest area is the content selected by the user; the user selects part of the content in the patent document drawings to be displayed, in which case, first, determine the content selected by the user. If the length of a line segment in the patent document drawings to be displayed within the selected area exceeds a preset ratio of the total length of the line segment, the line segment will be regarded as the content selected by the user. This application does not limit the preset ratio. If the preset ratio is higher than 50%, the content selected by the user when selecting the frame will be relatively small; if the preset ratio is lower than 50%, the content selected by the user when selecting the frame will be relatively large. Then, if the ratio of the number of segments to be highlighted in the content selected by the user exceeds the preset ratio, a cancel highlighting instruction is sent, otherwise, a highlighting instruction is sent. When the user selects, due to the coarse selection granularity of the selection, some other content will inevitably be selected, that is, when the user wants to select some content for highlighting, it is easy to select some content that has been highlighted. Therefore, it is necessary to judge whether the user wants to perform the highlighting / cancelling operation according to the proportion of the content that has been highlighted to the content selected by the user. Preferably, the preset ratio is 70%, that is, when the number of segments to be highlighted accounts for more than 70% of the number of segments to be selected, a cancel highlighting instruction is sent, otherwise, a highlighting instruction is sent. Under this condition, it is easier to perform the highlighting operation when the user selects. Of course, the highlighting / cancelling the highlighting instruction sending rule is only executed when the user selects but does not provide a clear operation instruction. If the user gives a clear operation instruction, the operation is performed according to the user's instruction.

[0097] Step S702: according to the highlighting / cancelling instruction of the content of the drawing, highlighting / cancelling the highlighting of the content of the drawing in the target drawing to be displayed.

[0098] In this embodiment, when the user highlights / unhighlights the content of the patent document drawings, the content of the patent document drawings will be highlighted / unhighlighted, so that the user can highlight certain contents of the patent document drawings or unhighlight certain contents of the drawings as needed, thereby facilitating the user to find the position of the mechanism corresponding to the current reading content in the patent document drawings to be displayed, thereby facilitating the user to quickly understand the technical solution provided by the current patent document.

[0099] In the method for displaying drawings of patent documents provided in the embodiment of the present application, the user may choose to look at the drawings of the specification first, roughly understand the entire technical solution, and then check the specification for the components that are of interest or not understood. In view of the above situation, the user can select the drawing number in the drawing in the drawing display area. The present application does not limit how the user performs the selection operation. It can be a left-click on the drawing number in the drawing display area, or select the drawing number in the drawing display area and click the right-click menu. After the user performs the selection operation, the text display area automatically jumps to the area where the selected drawing number appears for the first time in the specific implementation method part of the specification, and the content described in this area is often the specific connection method of the component corresponding to the drawing number selected by the user and the structural name of the component, and all the drawing numbers in the specification that are the same as the selected drawing number are highlighted, so as to facilitate the user to understand the specific connection method and structural name of the component corresponding to the selected drawing number, so as to facilitate the user to quickly understand the technical solution provided by the patent document currently being read.

[0100] Corresponding to the method of the above embodiment, Figure 8 A structural block diagram of a patent document illustration display device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 8 The exemplary patent document illustration display device may be the execution body of the patent document illustration display method provided in the aforementioned embodiment 1.

[0101] Reference Figure 8 , the device shown in the attached drawings of the patent document includes:

[0102] A patent document drawing display request receiving module 810 is used to receive a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers;

[0103] An initial drawing to be displayed determining module 820 is used to determine the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set;

[0104] An alignment feature generating module 830 is used to extract semantic features of the initial drawings to be displayed and the search text information based on a plurality of preset scale feature windows, and generate alignment features of the drawings to be displayed and alignment features of the search text;

[0105] The target drawing to be displayed is determined 840, which is used to match and analyze the alignment features of the drawing to be displayed with the alignment features of the search text, and determine the target drawing to be displayed for display to the user.

[0106] The patent document drawings provided in the embodiment of this application show the process of each module in the device realizing its own function. For details, please refer to the aforementioned Figure 1 The description of the first embodiment is not repeated here.

[0107] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0108] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0109] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0110] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0111] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or suggesting relative importance. It should also be understood that although the terms "first", "second", etc. are used to describe various elements in some embodiments of the present application in the text, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first table can be named as the second table, and similarly, the second table can be named as the first table without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0112] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0113] The patent document illustration display method provided in the embodiment of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0114] For example, the terminal device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set top box (STB), a customer premises equipment (CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, such as a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network, etc.

[0115] As an example but not limitation, when the terminal device is a wearable device, the wearable device can also be a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0116] Fig. 9 Schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Fig. 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Fig. 9 Only one is shown in the figure), a memory 91, wherein the memory 91 stores a computer program 92 that can be run on the processor 90. When the processor 90 executes the computer program 92, the steps in the method embodiments shown in the drawings of the above-mentioned patent documents are implemented, such as Figure 1 Alternatively, when the processor 90 executes the computer program 92, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 8 Functions of modules 810 to 840 are shown.

[0117] The terminal device 9 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will appreciate that Fig. 9 It is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include an input sending device, a network access device, a bus, etc.

[0118] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0119] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. The memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 9. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 91 may also be used to temporarily store data that has been sent or is to be sent.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] An embodiment of the present application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor, wherein when the processor executes the computer program, the terminal device implements the steps in any of the above-mentioned method embodiments.

[0122] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for displaying drawings of patent documents, characterized in that: include: Receive a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers; Determine the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set; Based on a plurality of preset scale feature windows, semantic features are extracted from the initial drawings to be displayed and the search text information respectively to generate alignment features of the drawings to be displayed and alignment features of the search text; Matching and analyzing the alignment features of the to-be-displayed drawings with the alignment features of the searched texts to determine the target to-be-displayed drawings for display to the user; According to the mutual inclusion relationship between the figure numbers of the drawings in the patent document and the set of figure numbers, the initial drawings to be displayed are determined, including: According to the mutual inclusion relationship between the figure numbers of the figures of the patent document and the figure number set, determining the figure of the patent document that contains the figure number set and has the least total number of figure numbers; According to the set of figure numbers, extracting the initial figure to be displayed from the figures of the patent document that contain the set of figure numbers and have the least total number of figure numbers; The method of extracting the initial drawings to be displayed from the drawings of the patent document that contain the set of drawing numbers and have the least total number of drawing numbers according to the set of drawing numbers includes: Determine the position of each figure number in the set of figure numbers and the position of each indicator line corresponding to the figure number in the figure number of the patent document figure that contains the set of figure numbers and has the least total number of figure numbers; Determine the endpoint coordinates of each indicator line away from the drawing number according to the position of each drawing number and the position of each indicator line corresponding to the drawing number; Determine the center coordinates of the drawing according to the coordinates of the endpoints of each indicator line away from the drawing number; According to the center coordinates of the drawings, the coordinates of the endpoints of the ends of the indicator lines away from the drawing numbers, and the preset lengths of the indicator lines, the positions of the target indicator lines corresponding to the drawing numbers are determined in the drawings of the patent document that contain the set of drawing numbers and have the least total number of drawing numbers, so that the extension lines of the target indicator lines intersect with the center coordinates of the drawings; Determining the position of the target figure number according to the positions of the target indication lines corresponding to the figure number; According to the position of the target figure number, extracting the initial figure to be displayed from the figures of the patent document that contain the figure number set and have the least total number of figure numbers; The step of extracting semantic features from the initial drawings to be displayed and the search text information based on a plurality of preset scale feature windows to generate alignment features of the drawings to be displayed and alignment features of the search text specifically includes: Performing a multi-scale convolution operation on a plurality of preset scale feature windows and the initial figure to be displayed to obtain a plurality of local representation feature maps of the figure to be displayed; Performing a multi-scale convolution operation on a plurality of preset scale feature windows and the search text information to obtain a plurality of search text local representation feature maps; Calculating the average value of all values ​​in the local representation feature graphs of the drawings to be displayed to obtain a plurality of global representation feature graphs of the drawings to be displayed; Calculate the average value of all values ​​in each of the local representation feature graphs of the search text to obtain multiple global representation feature graphs of the search text; Aggregation operations are performed on the global representation feature graph of the to-be-displayed figure and the global representation feature graph of the search text respectively to generate alignment features of the to-be-displayed figure and alignment features of the search text.

2. The method for displaying drawings of patent documents according to claim 1, characterized in that: The step of performing aggregation operations on the global representation feature graph of the to-be-displayed drawing and the global representation feature graph of the search text to generate the alignment feature of the to-be-displayed drawing and the alignment feature of the search text specifically includes: According to a preset image unit transformation matrix, a preset image-text matching parameter matrix, and a preset image-text feature mapping matrix, matrix operations are performed on the global representation feature maps of the plurality of drawings to be displayed to obtain alignment vectors of the drawings to be displayed; Performing weighted summation on the alignment vectors of the drawings to be displayed to obtain alignment features of the drawings to be displayed; According to a preset text unit transformation matrix, a preset image-text matching parameter matrix, and a preset image-text feature mapping matrix, a matrix operation is performed on a plurality of the retrieval text global representation feature maps to obtain a retrieval text alignment vector; A weighted sum is performed on the search text alignment vectors to obtain a search text alignment feature.

3. The method for displaying drawings of patent documents according to claim 1, characterized in that: The step of matching and analyzing the alignment features of the drawings to be displayed with the alignment features of the searched text to determine the target drawings to be displayed specifically includes: Calculating the Euclidean distance between the alignment feature of the to-be-displayed figure and the alignment feature of the searched text; Calculating the matching probability between the alignment feature of the to-be-displayed drawing and the alignment feature of the search text according to the Euclidean distance; When the matching probability is greater than a preset matching probability threshold, the initial drawing to be displayed corresponding to the alignment feature of the drawing to be displayed is determined as the target drawing to be displayed.

4. The method for displaying drawings of patent documents according to claim 1, characterized in that: After the step of matching and analyzing the alignment features of the to-be-displayed drawings with the alignment features of the searched texts to determine the target to-be-displayed drawings for display to the user, the method further includes: receiving a highlighting / cancelling highlighting instruction of the content of the drawing sent by the user based on the drawing to be displayed for the purpose; According to the highlighting / unchaiming instruction of the content of the drawing, the content of the drawing is highlighted / unchaiming in the target drawing to be displayed.

5. A patent document drawing display device, characterized in that: include: A patent document drawing display request receiving module, used to receive a patent document drawing display request from a user, wherein the patent document drawing display request includes search text information and a drawing number set, wherein the drawing number set consists of at least two drawing numbers; An initial drawing to be displayed determining module, used for determining the initial drawings to be displayed according to the mutual inclusion relationship between the drawing numbers of the drawings in the patent document and the drawing number set; An alignment feature generation module, configured to extract semantic features of the initial drawings to be displayed and the search text information based on a plurality of preset scale feature windows, and generate alignment features of the drawings to be displayed and alignment features of the search text; The target drawings to be displayed are determined, which is used to match and analyze the alignment features of the drawings to be displayed with the alignment features of the search text, and determine the target drawings to be displayed for display to the user; According to the mutual inclusion relationship between the figure numbers of the drawings in the patent document and the set of figure numbers, the initial drawings to be displayed are determined, including: According to the mutual inclusion relationship between the figure numbers of the figures of the patent document and the figure number set, determining the figure of the patent document that contains the figure number set and has the least total number of figure numbers; According to the set of figure numbers, extracting the initial figure to be displayed from the figures of the patent document that contain the set of figure numbers and have the least total number of figure numbers; The method of extracting the initial drawings to be displayed from the drawings of the patent document that contain the set of drawing numbers and have the least total number of drawing numbers according to the set of drawing numbers includes: Determine the position of each figure number in the set of figure numbers and the position of each indicator line corresponding to the figure number in the figure number of the patent document figure that contains the set of figure numbers and has the least total number of figure numbers; Determine the endpoint coordinates of each indicator line away from the drawing number according to the position of each drawing number and the position of each indicator line corresponding to the drawing number; Determine the center coordinates of the drawing according to the coordinates of the endpoints of each indicator line away from the drawing number; According to the center coordinates of the drawings, the coordinates of the endpoints of the ends of the indicator lines away from the drawing numbers, and the preset lengths of the indicator lines, the positions of the target indicator lines corresponding to the drawing numbers are determined in the drawings of the patent document that contain the set of drawing numbers and have the least total number of drawing numbers, so that the extension lines of the target indicator lines intersect with the center coordinates of the drawings; Determining the position of the target figure number according to the positions of the target indication lines corresponding to the figure number; According to the position of the target figure number, extracting the initial figure to be displayed from the figures of the patent document that contain the figure number set and have the least total number of figure numbers; The step of extracting semantic features from the initial drawings to be displayed and the search text information based on a plurality of preset scale feature windows to generate alignment features of the drawings to be displayed and alignment features of the search text specifically includes: Performing a multi-scale convolution operation on a plurality of preset scale feature windows and the initial figure to be displayed to obtain a plurality of local representation feature maps of the figure to be displayed; Performing a multi-scale convolution operation on a plurality of preset scale feature windows and the search text information to obtain a plurality of search text local representation feature maps; Calculating the average value of all values ​​in the local representation feature graphs of the drawings to be displayed to obtain a plurality of global representation feature graphs of the drawings to be displayed; Calculate the average value of all values ​​in each of the local representation feature graphs of the search text to obtain multiple global representation feature graphs of the search text; Aggregation operations are performed on the global representation feature graph of the to-be-displayed figure and the global representation feature graph of the search text respectively to generate alignment features of the to-be-displayed figure and alignment features of the search text.

6. A terminal device, characterized in that: The terminal device includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images