Method, device, equipment, and storage medium for processing non-editable documents

Abstract

The present application discloses a method and apparatus, device, and storage medium for processing a non-editable document; wherein the method comprises: identifying a target object in a non-editable document and generating a first editable frame of the target object; displaying the first editable frame at the position of the target object; adjusting the first editable frame based on a first operation received by the first editable frame to obtain a second editable frame; and generating an editable object from the target object in the second editable frame.

Description

Technical Field

[0001] The embodiments of the present application relate to electronic technology, and relate to but are not limited to a method and apparatus, device, and storage medium for processing non-editable documents. Background Art

[0002] When recognizing objects in non-editable documents, in many cases the objects in the document cannot be completely recognized correctly. In this case, after the recognition error occurs, the user generally corrects it by manually modifying the recognition result, which is costly. Summary of the invention

[0003] In view of this, the method, apparatus, device, and storage medium for processing non-editable documents provided in the embodiments of the present application are implemented as follows:

[0004] According to one aspect of an embodiment of the present application, a method for processing an uneditable document is provided, including: identifying a target object in the uneditable document and generating a first editable frame of the target object; displaying the first editable frame at the position of the target object; adjusting the first editable frame based on a first operation received by the first editable frame to obtain a second editable frame; and generating an editable object from the target object in the second editable frame.

[0005] According to one aspect of an embodiment of the present application, a device for processing a non-editable document is provided, including: a generation module, used to identify a target object in the non-editable document and generate a first editable frame of the target object; a display module, used to display the first editable frame at the position of the target object; an adjustment module, used to adjust the first editable frame based on a first operation received by the first editable frame to obtain a second editable frame; the generation module is also used to generate an editable object from the target object in the second editable frame.

[0006] According to one aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the method described in the embodiment of the present application is implemented.

[0007] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0008] In the embodiment of the present application, when the user is not satisfied with the recognition result of the target object, the user does not adjust the recognition result of the target object, but adjusts the first editable frame of the target object to obtain a second editable frame with a better presentation effect, and then re-recognizes the target object in the second editable frame through the electronic device to obtain the final editable object (i.e., the new recognition result). In this way, the problem of high modification cost caused by the user manually modifying the recognition result is effectively avoided, thereby reducing the error correction cost and improving the user experience.

[0009] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0012] Figure 1 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in an embodiment of the present application;

[0013] Figure 2 A schematic diagram of an implementation method for adjusting a first editable box provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of another implementation method for adjusting the first editable box provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of a target object corresponding to multiple candidate detection frames provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in an embodiment of the present application;

[0018] Figure 7A schematic diagram of the implementation flow of the method for processing non-editable documents provided in an embodiment of the present application;

[0019] Figure 8 A schematic diagram of a display method of the edge information of a candidate detection box provided in an embodiment of the present application;

[0020] Fig. 9 A schematic diagram of adjusting a first editable box based on side information of a candidate detection box provided in an embodiment of the present application;

[0021] Fig.10 A schematic diagram of a formula detection and recognition result provided in an embodiment of the present application;

[0022] Fig.11 A schematic diagram of another formula detection and recognition result provided in an embodiment of the present application;

[0023] Fig.12 A schematic diagram of the structure of a device for processing non-editable documents provided in an embodiment of the present application;

[0024] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0027] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application do not represent a specific ordering of objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0029] The embodiment of the present application provides a method for processing a non-editable document, which is applied to an electronic device, which can be various types of devices with information processing capabilities during implementation, for example, the electronic device can include a personal computer (such as a desktop, a laptop, a small laptop, a tablet computer, etc.), a mobile phone, a personal digital assistant (PDA), etc. The function implemented by the method can be implemented by calling a program code by a processor in the electronic device, and of course the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0030] Figure 1 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include the following steps 101 to 104:

[0031] Step 101: Identify a target object in a non-editable document and generate a first editable frame of the target object.

[0032] In the embodiments of the present application, there is no limitation on the form of the non-editable document. For example, the non-editable document is a picture containing a target object. Another example is a non-editable document is a file that cannot be edited, such as a portable document format (PDF).

[0033] In the embodiments of the present application, there is no limitation on the type of the target object, which can be of various types. In some embodiments, the target object includes a formula.

[0034] In some embodiments, step 101 may be implemented by executing steps 401 to 403 in the following embodiments.

[0035] Step 102, displaying a first editable box at the location of the target object;

[0036] Step 103: Based on the first operation received by the first editable box, adjust the first editable box to obtain a second editable box.

[0037] In some embodiments, the first operation may be an operation indicating dragging the first editable box or dragging a specific side of the first editable box; accordingly, adjusting the first editable box may be adjusting the entire first editable box or only adjusting a specific side of the first editable box.

[0038] For example, if Figure 2As shown, in the case where the first operation is used to instruct to move the position of the first editable box 201 to the first designated position, in response to the first operation, the first editable box 201 is moved to the first designated position to obtain the second editable box 202. In this way, the adjusted second editable box includes the complete content of the target object, so that when the target object in the second editable box is subsequently re-identified, the generated editable object is the complete target object.

[0039] For example, Figure 3 As shown, in the case where the first operation is used to instruct to adjust the specific side 302 of the first editable frame 301 to the second specified position, in response to the first operation, the first editable frame 301 is adjusted based on the specific side 302 and the second specified position to obtain the second editable frame 303. In implementation, the user can adjust the specific side 302 to the second specified position that completely includes the target object by pulling it alone, thereby obtaining the second editable frame 303. Of course, the user can also continue to pull other sides of the first editable frame until the adjusted second editable frame completely includes the target object.

[0040] In some other embodiments, the first operation may also indicate presenting auxiliary information as shown in step 405 in the following embodiments, so that the user can adjust the first editable box according to the presented auxiliary information.

[0041] Step 104: Generate an editable object from the target object in the second editable frame.

[0042] Here, after the target object in the second editable frame is generated into an editable object, the user can perform operations such as rearrangement or copying the editable object to continue using the target object in other places.

[0043] In the embodiment of the present application, when the user is not satisfied with the recognition result of the target object, the user does not adjust the recognition result of the target object, but adjusts the first editable frame of the target object to obtain a second editable frame with a better presentation effect, and then re-recognizes the target object in the second editable frame through the electronic device to obtain the final editable object (i.e., the new recognition result). In this way, the problem of high modification cost caused by the user manually modifying the recognition result is effectively avoided, thereby reducing the error correction cost and improving the user experience.

[0044] Figure 4 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in the embodiment of the present application is shown in FIG. Figure 4 As shown, the method may include the following steps 401 to 407:

[0045] Step 401 : Detect a target object in a non-editable document and obtain a plurality of candidate detection boxes of the target object.

[0046] like Figure 5 As shown, for target object 1 (i.e., calculation 1), multiple candidate detection frames are located at different positions in the same area of ​​target object 1, and these candidate detection frames respectively contain part of target object 1 or all of target object 1. In addition, if target object 1 (i.e., calculation 1) and target object 2 (i.e., calculation 2) are close, then when detecting target object 1, its corresponding candidate detection frame 501 may also contain part of the content of target object 2.

[0047] Step 402: Determine the confidence level of each candidate detection box.

[0048] Here, the confidence of the candidate detection frame is used to characterize the probability that the candidate detection frame completely contains the target object. It can be understood that the higher the confidence, the greater the probability that the candidate detection frame completely contains the target object.

[0049] Step 403: Generate a first editable frame based on the candidate detection frames whose confidences meet the conditions among the multiple candidate detection frames.

[0050] In the embodiments of the present application, there is no limitation on the implementation method of generating the first editable box. For example, in some embodiments, based on the confidences of multiple candidate detection boxes, the candidate detection box with the highest confidence can be determined as the first editable box. For example, in other embodiments, the candidate detection boxes whose confidences meet the conditions among multiple candidate detection boxes can be merged to generate the first editable box. Here, the candidate detection boxes whose confidences meet the conditions can be the first N candidate detection boxes with high confidence ranking, or can be M candidate detection boxes whose confidences meet the threshold (such as confidence greater than 0.8), and there is no limitation on this.

[0051] Step 404, displaying a first editable box at the location of the target object;

[0052] Step 405, when the first operation received by the first editable box is used to indicate the presentation of auxiliary information, in response to the first operation, the auxiliary information of the first editable box is displayed at the position of the target object; wherein the auxiliary information is used to assist in adjusting the first editable box.

[0053] Here, when displaying the auxiliary information of the first editable box, the display method is not limited. For example, after receiving the first operation, the auxiliary information can be directly displayed at the position of the target object; or, after receiving the display instruction, the auxiliary information can be displayed. Among them, the method of receiving the display instruction is also not limited, such as displaying a control button at the position of the target object; when it is detected that the control button receives a click operation, it is determined that the display instruction is received; or, when the user double-clicks the first editable box, it is determined that the display instruction is received.

[0054] In the embodiment of the present application, when the user determines that the effect of the target object contained in the first editable box is not satisfactory, the user does not adjust the recognition result of the target object in the first editable box, but initiates a first operation on the first editable box to display auxiliary information of the first editable box at the position of the target object, so that the user can adjust the first editable box based on the auxiliary information. In this way, compared with directly adjusting the recognition result, it is easier for the user to adjust the editable box, thereby reducing the error correction cost and providing a better user experience.

[0055] Step 406: Based on the second operation received from the auxiliary information, adjust the first editable box to obtain a second editable box.

[0056] In an embodiment of the present application, when a user wants to adjust the first editable box, displaying auxiliary information of the first editable box can help the user quickly determine the adjustment method for the first editable box, so that the second editable box can be obtained more quickly.

[0057] In some embodiments, step 406 may be implemented by executing step 607 or step 707 in the following embodiments. In this way, after the first editable frame is adjusted to the second editable frame, the target object contained in the second editable frame is a complete target object, so that the editable object generated based on the target object in the second editable frame is also a complete editable object for subsequent use by the user.

[0058] Step 407: Generate an editable object from the target object in the second editable frame.

[0059] Figure 6 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the method may include the following steps 601 to 608:

[0060] Step 601, detecting a target object in a non-editable document, and obtaining a plurality of candidate detection boxes of the target object;

[0061] Step 602, determining the confidence of each candidate detection box;

[0062] Step 603, generating a first editable frame based on the candidate detection frames whose confidences meet the conditions among the multiple candidate detection frames;

[0063] Step 604, displaying a first editable box at the location of the target object;

[0064] Step 605 , generating a candidate object from the target object in the candidate detection frame; and using at least one candidate object of the generated target object as auxiliary information.

[0065] Here, after obtaining multiple candidate detection frames corresponding to the target object, the target objects in some of the candidate detection frames (such as the top N candidate detection frames with high confidence rankings, or M candidate detection frames whose confidences meet a threshold) can be generated as candidate objects, or the target objects in all the candidate detection frames can be generated as candidate objects, and there is no limitation on this.

[0066] Step 606: When the first editable box receives a first operation, in response to the first operation, display at least one candidate object of the target object at the position of the target object; wherein the first operation is used to instruct the presentation of auxiliary information.

[0067] In some embodiments, after at least one candidate object of the target object is displayed as auxiliary information, an object that the user is more satisfied with can be selected from the candidate objects as the final editable object. In this way, the user can directly select a satisfactory object from the candidate objects without adjusting the first editable box or performing other complicated operations, thereby further saving the user's operation cost.

[0068] Step 607: Based on the candidate object receiving the second operation, adjust the first editable box to obtain a second editable box.

[0069] In the embodiment of the present application, there is no limit on the number of candidate objects that receive the second operation. The second operation may be received by one candidate object or by multiple candidate objects.

[0070] There is no limitation on the method of obtaining the second editable frame. For example, the candidate object for receiving the second operation and the object in the first editable frame may be merged, and the second editable frame may be generated based on the merged object; the position to be adjusted of the first editable frame may be determined based on the positions of the displayed multiple candidate objects, so as to adjust the first editable frame and obtain the second editable frame; the second editable frame may also be generated based on the candidate detection frame corresponding to the candidate object for receiving the second operation and the first editable frame.

[0071] Step 608: Generate an editable object from the target object in the second editable frame.

[0072] In an embodiment of the present application, when the user is not satisfied with the effect of the target object contained in the first editable box, the user does not adjust the recognition result of the target object in the first editable box, but initiates a first operation on the first editable box to display at least one candidate object of the target object at the position of the target object, thereby adjusting the first editable box based on the candidate object to obtain the second editable box. In this way, on the one hand, compared with directly adjusting the recognition result, it is easier for the user to adjust the editable box, thereby reducing the error correction cost and providing a better user experience; on the other hand, by displaying at least one candidate object of the target object as auxiliary adjustment information, it can help the user to quickly determine the adjustment method for the first editable box, so that the second editable box can be obtained more quickly.

[0073] Figure 7 A schematic diagram of the implementation flow of the method for processing non-editable documents provided in the embodiment of the present application is shown in FIG. Figure 7 As shown, the method may include the following steps 701 to 708:

[0074] Step 701, detecting a target object in a non-editable document, and obtaining a plurality of candidate detection boxes of the target object;

[0075] Step 702, determining the confidence of each candidate detection box;

[0076] Step 703, generating a first editable frame based on the candidate detection frames whose confidences meet the conditions among the multiple candidate detection frames;

[0077] Step 704, displaying a first editable box at the location of the target object;

[0078] Step 705: Use the side information of at least one candidate detection box except the first editable box as auxiliary information.

[0079] Here, displaying the edge information of the candidate detection box may be displaying all edges of the candidate detection box, or may be displaying one or more edges (less than the total number of edges) of the candidate detection box, and there is no limitation on this.

[0080] Step 706, when the first editable box receives the first operation, in response to the first operation, display the side information of at least one candidate detection box other than the first editable box at the position of the target object; wherein the first operation is used to indicate the presentation of auxiliary information.

[0081] In some embodiments, when a first operation is used to indicate adjustment of a first side of a first editable box, in response to the first operation, a second side having the same orientation as the first side in at least one candidate detection box other than the first editable box is displayed at the position of the target object.

[0082] For example, if Figure 8 As shown, when the first operation is used to indicate adjustment of the right side of the first editable box, the corresponding display is also the right side of the candidate detection box with the same orientation, without displaying all the sides of the candidate detection box, thereby avoiding confusing the user's vision when there are too many sides.

[0083] Step 707: Based on the side information of the candidate detection box selected by the second operation, adjust the first editable box to obtain a second editable box.

[0084] Here, there is no limitation on the adjustment method based on the edge information of the candidate detection frame. Fig. 9 As shown, for example, when the side information of the candidate detection frame indicates that the first editable frame is smaller, the diagonal line of the first editable frame can be pulled to adjust it based on the side information of the candidate detection frame, or the sides of the first editable frame can be pulled separately to adjust it to the position of the second side of the selected candidate detection frame, thereby obtaining the second editable frame. For another example, when the side information of the candidate detection frame indicates that the right side of the first editable frame needs to be adjusted, the right side (first side) of the first editable frame can be pulled to the position of the second side of the selected candidate detection frame based on the side information of the candidate detection frame, thereby obtaining the second editable frame.

[0085] Step 708: Generate an editable object from the target object in the second editable frame.

[0086] In an embodiment of the present application, when the user is not satisfied with the effect of the target object contained in the first editable frame, the user does not adjust the recognition result of the target object in the first editable frame, but initiates a first operation on the first editable frame to display the side information of at least one candidate detection frame at the position of the target object, thereby adjusting the first editable frame based on the side information of the candidate detection frame to obtain the second editable frame. In this way, on the one hand, compared with directly adjusting the recognition result, it is easier for the user to adjust the editable frame, which reduces the error correction cost and provides a better user experience; on the other hand, by displaying the side information of at least one candidate detection frame of the target object as auxiliary adjustment information, it can help the user to quickly determine the adjustment method for the first editable frame, so that the second editable frame can be obtained more quickly.

[0087] In the process of recognizing printed formulas (a form of expression of the target object), the correctness of the recognition result depends to a large extent on the accuracy of the formula detection. The accuracy of the current printed formula detection is evaluated based on the Intersection over Union (IoU). In practice, a detection box with a high IoU does not completely represent the accuracy of the detection and may still lead to recognition errors. If the recognition result is wrong due to formula detection, the user needs to spend a lot of money to manually modify it again.

[0088] For example, as shown in 10, for the formula 1001 For example, its corresponding IOU value is 0.88, but the detection box does not completely contain π in -2π. In this way, although the IOU value of formula 1001 is very high, the formula content cannot be fully recognized during the subsequent recognition of the formula in the detection box, and the recognition result may be wrong, such as failing to recognize π or misidentifying π as t.

[0089] For example, Fig.11 As shown, for formula 111, its corresponding detection box does not completely contain it. Therefore, although the IOU value corresponding to the detection box is 0.84, when the formula in the detection box is subsequently recognized, it is very likely that the formula content cannot be fully recognized, resulting in an incorrect recognition result.

[0090] Based on this, in an embodiment of the present application, an algorithm is used to provide some alternative recognition results or detection frame adjustment prompts to assist users in editing and adjusting the detection and recognition results, thereby enabling the detection and recognition results to be adjusted more conveniently and quickly.

[0091] In an embodiment of the present application, a method for assisting in adjusting a formula to detect a recognition result may be implemented by the following steps 1 to 6:

[0092] Step 1: First, use the formula detection model to detect the formula in the image and retain multiple detection box positions of different sizes at the same position.

[0093] Step 2: Use the formula detection model to identify the results of detection boxes of different sizes and save multiple recognition result candidates.

[0094] Step 3: Output the detection and recognition result with the highest confidence to the user.

[0095] Step 4: When the user needs to edit a certain recognition result, the previously stored recognition result candidates are preferentially displayed for the user to select.

[0096] Step 5: If the user attempts to adjust the detection result box, the previously saved detection box position is displayed to the user as an auxiliary reference line to assist the user in the adjustment when the user drags the box.

[0097] Step 6: Output the final result according to the user's modification.

[0098] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0099] Based on the foregoing embodiments, the embodiments of the present application provide a device for processing non-editable documents, which includes the modules included and the units included in the modules, and can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0100] Fig.12 Schematic diagram of the structure of the device for processing non-editable documents according to the embodiment of the present application. Fig.11 As shown, the device 120 includes a generating module 121, a display module 122 and an adjusting module 123, wherein:

[0101] A generation module 121 is used to identify a target object in a non-editable document and generate a first editable frame for the target object; a display module 122 is used to display the first editable frame at the position of the target object; an adjustment module 123 is used to adjust the first editable frame based on a first operation received by the first editable frame to obtain a second editable frame; the generation module 121 is also used to generate an editable object from the target object in the second editable frame.

[0102] In some embodiments, the device 120 also includes a response module, which is used to move the first editable box to the first specified position in response to the first operation to obtain the second editable box when the first operation is used to indicate moving the position of the first editable box to a first specified position; the response module is also used to adjust the first editable box based on the specific edge and the second specified position in response to the first operation to obtain the second editable box when the first operation is used to indicate adjusting a specific edge of the first editable box to a second specified position.

[0103] In some embodiments, the response module is used to display auxiliary information of the first editable box at the position of the target object in response to the first operation when the first operation is used to indicate the presentation of auxiliary information; wherein the auxiliary information is used to assist in adjusting the first editable box; the adjustment module 123 is also used to adjust the first editable box based on a second operation received based on the auxiliary information to obtain a second editable box.

[0104] In some embodiments, the device 120 also includes a detection module and a determination module, the detection module is used to detect the target object in the non-editable document and obtain multiple candidate detection boxes of the target object; the determination module is used to determine the confidence of each of the candidate detection boxes; the generation module 121 is used to generate the first editable box based on the candidate detection boxes whose confidences meet the conditions among the multiple candidate detection boxes.

[0105] In some embodiments, a generation module 121 is used to generate a candidate object from the target object in the candidate detection frame; and use at least one candidate object of the generated target object as the auxiliary information; an adjustment module 123 is used to adjust the first editable frame based on the candidate object received for the second operation to obtain the second editable frame.

[0106] In some embodiments, the adjustment module 123 is used to adjust the first editable box based on the side information of the candidate detection box selected by the second operation to obtain the second editable box.

[0107] In some embodiments, the response module is further used to display a second edge in the same orientation as the first edge in at least one of the candidate detection frames other than the first editable frame at the position of the target object in response to the first operation.

[0108] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0109] It should be noted that in the embodiments of this application Fig.12 The division of modules in the processing device of the non-editable document shown is schematic and is only a logical functional division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0110] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0111] An embodiment of the present application provides an electronic device, Fig.13 Schematic diagram of the hardware entity of the electronic device of the embodiment of the present application, such as Fig.13 As shown, the electronic device 130 includes a memory 131 and a processor 132, wherein the memory 131 stores a computer program that can be run on the processor 132, and the processor 132 implements the steps in the method provided in the above embodiment when executing the program.

[0112] It should be noted that the memory 131 is configured to store instructions and applications executable by the processor 132, and can also cache data to be processed or processed by the processor 132 and various modules in the electronic device 130 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0113] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiment are implemented.

[0114] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the method provided in the above method embodiment.

[0115] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0116] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0117] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can mean: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0118] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0120] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0122] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0123] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0124] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0125] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0126] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0127] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing a non-editable document, characterized in that: The method comprises: Identify a target object in a non-editable document, and generate a first editable frame of the target object; Displaying the first editable frame at the location of the target object; In a case where the first operation is used to instruct to present auxiliary information, in response to the first operation, auxiliary information of the first editable box is displayed at the position of the target object; wherein the auxiliary information is used to assist in adjusting the first editable box; the auxiliary information includes side information of at least one candidate detection box other than the first editable box; the number of sides of the side information of the one candidate detection box is less than the total number of sides of the candidate detection box; Based on the side information of the candidate detection box selected by the second operation, adjusting the first editable box to obtain the second editable box; and The target object in the second editable frame is generated into an editable object.

2. The method according to claim 1, characterized in that The adjusting the first editable box based on the first operation received by the first editable box to obtain a second editable box includes: In a case where the first operation is used to instruct to move the position of the first editable box to a first specified position, in response to the first operation, the first editable box is moved to the first specified position to obtain the second editable box; When the first operation is used to instruct to adjust a specific side of the first editable box to a second specified position, in response to the first operation, the first editable box is adjusted based on the specific side and the second specified position to obtain the second editable box.

3. The method according to claim 1, characterized in that The step of identifying a target object in a non-editable document and obtaining a first editable frame of the target object includes: Detecting a target object in the non-editable document to obtain a plurality of candidate detection frames of the target object; Determining the confidence of each of the candidate detection boxes; The first editable frame is generated based on the candidate detection frames whose confidences meet the conditions among the multiple candidate detection frames.

4. The method according to claim 3, characterized in that The method further comprises: Generating a candidate object from the target object in the candidate detection frame; and using at least one candidate object of the generated target object as the auxiliary information; Accordingly, the second operation received based on the auxiliary information adjusts the first editable box to obtain a second editable box, including: Based on the candidate object receiving the second operation, the first editable box is adjusted to obtain the second editable box.

5. The method according to claim 1, characterized in that The first operation is used to instruct to adjust the first side of the first editable box; The step of displaying auxiliary information of the first editable box at the location of the target object in response to the first operation includes: In response to the first operation, a second side in at least one of the candidate detection frames other than the first editable frame and having the same orientation as the first side is displayed at the position of the target object.

6. The method according to any one of claims 1 to 5, characterized in that: The target object includes a formula.

7. A device for processing a non-editable document, comprising: a response module, configured to display auxiliary information of the first editable box at the position of the target object in response to the first operation when the first operation is used to indicate the presentation of auxiliary information; wherein the auxiliary information is used to assist in adjusting the first editable box; the auxiliary information includes side information of at least one candidate detection box other than the first editable box; and the number of sides of the side information of the one candidate detection box is less than the total number of sides of the candidate detection box; A generating module, used for identifying a target object in a non-editable document and generating a first editable frame of the target object; A display module, configured to display the first editable frame at the location of the target object; an adjusting module, configured to adjust the first editable box based on the side information of the candidate detection box selected by the second operation to obtain the second editable box; The generating module is further used to generate an editable object from the target object in the second editable frame.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images