Writed document processing method and device, electronic equipment and computer readable medium

Abstract

The invention relates to a written document processing method and device, electronic equipment and a computer readable medium. The method comprises the following steps: acquiring an image-text control identifier in a preset area; determining a target interaction instruction and a target interaction object area based on the image-text control identifier; obtaining a document image of a written document in the target interaction object area; and processing the document image based on the target interaction instruction. According to the written document processing method and device, the electronic equipment and the computer readable medium, the multimedia interaction function of the written document of the user can be realized without depending on a display screen, and the user can also call computer multimedia resources for interaction.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer information processing, and in particular, to a written document processing method and device, electronic equipment and computer readable medium.

[0002] The present application is a divisional application of the patent application with the application number 202011116909.5, the application date October 19, 2020, and the invention name written document processing method, device, electronic equipment and computer readable medium. BACKGROUND

[0003] Multimedia, online courses, remote office and other online work methods are being used more and more widely. Since the traditional online work method relies heavily on the display screen, it has a greater impact on the user's vision, especially not conducive to the eye health of children. Although products such as dot reading pens and smart pens using dot code technology can be used on paper documents to obtain multimedia interactive information, they need to be used with specially printed paper books, which requires high printing costs.

[0004] Therefore, there is a need for a new written document processing method, device, electronic equipment and computer readable medium.

[0005] The above information disclosed in the background section is only intended to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] Therefore, the present disclosure provides a written document processing method, device, electronic equipment and computer readable medium, which can realize the multimedia interaction function of the user's written document without relying on the display screen, and the user can also call computer multimedia resources for interaction.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to an aspect of the present disclosure, a written document processing method is provided, which comprises: acquiring a graphic-text control identifier in a preset area; determining a target interaction instruction and a target interaction object area based on the graphic-text control identifier; acquiring a document image of a written document in the target interaction object area; and processing the document image based on the target interaction instruction.

[0009] In an exemplary embodiment of the present disclosure, it further comprises: generating an interaction instruction mapping rule based on one or more graphic-text control identifiers and their corresponding interaction instructions; and / or generating an auxiliary control instruction mapping rule based on one or more graphic-text control identifiers and their corresponding auxiliary control instructions.

[0010] In an example embodiment of the present disclosure, the method further comprises: obtaining a graphic-text control identifier in a preset area; extracting a target auxiliary control instruction from an auxiliary control instruction mapping rule based on the graphic-text control identifier; and controlling a processing procedure or a multimedia device participating in the processing procedure based on the target auxiliary control instruction.

[0011] In an example embodiment of the present disclosure, the target auxiliary control instruction is generated based on the graphic-text control identifier and the auxiliary control instruction mapping rule, and the method further comprises: when the graphic-text control identifier includes a to-be-input parameter, inputting parameter information according to a preset manner, the parameter including at least one of voice data, text data, image data, and gesture data.

[0012] In an example embodiment of the present disclosure, the graphic-text control identifier in the preset area is obtained by: obtaining an object image of an identifier object placed in the preset area, the preset area being located in an imaging range of an imaging device; and performing image recognition on the object image of the identifier object to generate the graphic-text control identifier.

[0013] In an example embodiment of the present disclosure, the image recognition is performed on the object image of the identifier object to generate the graphic-text control identifier, and the method further comprises: performing image recognition on the object image of the identifier object to generate a recognition result; and eliminating invalid identifier objects in the recognition result to generate the graphic-text control identifier.

[0014] In an example embodiment of the present disclosure, the target interaction instruction and the target interaction object area are determined based on the graphic-text control identifier, and the method further comprises: extracting a target interaction instruction from the interaction instruction mapping rule according to the graphic-text control identifier; and determining the target interaction object area based on the target interaction instruction.

[0015] In an example embodiment of the present disclosure, the document image is processed based on the target interaction instruction, and the method further comprises: performing image recognition on the document image based on the target interaction instruction to generate the document information; and performing data processing on the document information.

[0016] In an example embodiment of the present disclosure, the document image is processed based on the target interaction instruction, and the method further comprises: performing image recognition on the document image based on the target interaction instruction to generate the document information; and performing data processing on the document information.

[0017] In an example embodiment of the present disclosure, processing the document image based on the target interaction instruction further includes: when the graphic-text control identifies a need for input parameters, inputting parameter information according to a preset manner, the parameters including at least one of voice data, text data, image data, and gesture data.

[0018] In an example embodiment of the present disclosure, further including: generating multimedia information based on the processing result; and performing a multimedia operation based on the multimedia information.

[0019] In an example embodiment of the present disclosure, performing a multimedia operation based on the multimedia information includes: determining a display area based on the graphic-text control; and projecting the processing result on the display area.

[0020] In an example embodiment of the present disclosure, further including: correcting the document image and / or the projected image projected on the display area.

[0021] In an example embodiment of the present disclosure, correcting the document image and / or the projected image projected on the display area includes: obtaining a first relative position of a known point on a written document corresponding to the document image; obtaining a second relative position of the known point in the document image; determining a distortion correction mapping between the written document and the document image based on the first relative position and the second relative position; and correcting the document image based on the distortion correction mapping.

[0022] In an example embodiment of the present disclosure, correcting the document image and / or the projected image projected on the display area further includes: superimposing the projected image on the document image after correction to generate a superimposed image; determining a deviation adjustment mapping of the projected image based on the superimposed image; and correcting the projected image based on the deviation adjustment mapping.

[0023] According to an aspect of the present disclosure, a method for processing a written document is provided, the method including: obtaining a graphic-text control identifier from a graphic-text control identifier area; obtaining a document image of a written document from an interactive object area; determining a target interaction instruction based on the graphic-text control identifier; and processing the document image based on the target interaction instruction.

[0024] In an example embodiment of the present disclosure, further including: generating at least one interactive instruction mapping rule based on one or more graphic-text control identifiers, at least one graphic-text control identifier area, and corresponding interactive instructions thereof; and / or generating at least one auxiliary control instruction mapping rule based on one or more graphic-text control identifiers, at least one graphic-text control auxiliary area, and corresponding auxiliary control instructions thereof.

[0025] In an example embodiment of the present disclosure, the method further comprises: obtaining a graphic control identifier by the graphic control auxiliary area; extracting a target auxiliary control instruction from the at least one auxiliary control instruction mapping rule based on the graphic control identifier and the number of the graphic control auxiliary area corresponding to the graphic control identifier; and controlling a processing procedure or a multimedia device participating in the processing procedure based on the target auxiliary control instruction.

[0026] In an example embodiment of the present disclosure, the extracting a target auxiliary control instruction from the at least one auxiliary control instruction mapping rule based on the graphic control identifier and the number of the graphic control auxiliary area corresponding to the graphic control identifier further comprises: when the graphic control identifier includes an input parameter, inputting parameter information according to a preset manner, the parameter including at least one of voice data, text data, image data, and gesture data.

[0027] In an example embodiment of the present disclosure, the method further comprises: adjusting a region range of the graphic control identifier area and / or the interactive object area and / or the graphic control auxiliary area; and wherein the graphic control identifier area, the interactive object area, and the graphic control auxiliary area are located within an imaging range of an imaging device.

[0028] In an example embodiment of the present disclosure, the adjusting a region range of the graphic control identifier area and / or the interactive object area and / or the graphic control auxiliary area comprises: obtaining an object image of an object for marking a region range placed within the imaging range; performing image recognition on the object image to generate a graphic control identifier for marking a region range; determining a target region range according to the graphic control identifier for marking a region range; and adjusting the region range of the interactive object area and / or the graphic control identifier area and / or the graphic control auxiliary area based on the target region range.

[0029] In an example embodiment of the present disclosure, the obtaining a graphic control identifier by the graphic control identifier area comprises: determining a number of the graphic control identifier area by a system setting; and obtaining the graphic control identifier from all of the graphic control identifier areas.

[0030] In an example embodiment of the present disclosure, the obtaining a graphic control identifier from all of the graphic control identifier areas comprises: obtaining an object image of an object for marking a region range placed in all of the graphic control identifier areas; and performing image recognition on the object image of the object for marking a region range to generate the graphic control identifier.

[0031] In an example embodiment of the present disclosure, the determining a target interactive instruction based on the graphic control identifier comprises: determining the target interactive instruction based on the graphic control identifier and the number of the graphic control identifier area corresponding to the graphic control identifier.

[0032] In an example embodiment of the present disclosure, determining the target interaction instruction based on the graphic-text control identifier and the number of the graphic-text control identifier area comprises: determining a target interaction instruction mapping rule from the at least one interaction instruction mapping rule based on the number of the graphic-text control identifier area; and extracting the target interaction instruction from the target interaction instruction mapping rule according to the graphic-text control identifier.

[0033] In an example embodiment of the present disclosure, processing the document image based on the target interaction instruction comprises: performing image recognition on the document image based on the target interaction instruction to generate the document information; and performing data processing on the document information.

[0034] In an example embodiment of the present disclosure, processing the document image based on the target interaction instruction to generate the document information comprises: determining a target area based on the target interaction instruction; and performing image recognition on the image in the target area in the document image to generate the document information.

[0035] In an example embodiment of the present disclosure, further comprising: generating multimedia information based on the processing result; and performing multimedia operation based on the multimedia information.

[0036] In an example embodiment of the present disclosure, performing multimedia operation based on the multimedia information comprises: determining a display area based on the graphic-text control identifier; and projecting the processing result on the display area.

[0037] In an example embodiment of the present disclosure, further comprising: correcting the document image and / or the projection image projected on the display area.

[0038] In an example embodiment of the present disclosure, correcting the document image and / or the projection image projected on the display area comprises: obtaining a first relative position of a known point on a written document corresponding to the document image; obtaining a second relative position of the known point in the document image; determining a distortion correction mapping between the written document and the document image based on the first relative position and the second relative position; and correcting the document image based on the distortion correction mapping.

[0039] In an example embodiment of the present disclosure, correcting the document image and / or the projection image projected on the display area further comprises: generating a superimposed image by superimposing the projection image on the document image after correction; determining a deviation adjustment mapping of the projection image according to the superimposed image; and correcting the projection image based on the deviation adjustment mapping.

[0040] According to an aspect of the present disclosure, a written document processing system is provided, which includes: an identification object configured to display a graphic-text control identification; an imaging device configured to acquire the graphic-text control identification in a preset area, and / or acquire a document image of a written document in a target interactive object area; a processing device configured to determine a target interactive instruction and the target interactive object area based on the graphic-text control identification, and process the document image based on the target interactive instruction; and / or determine a target auxiliary control instruction based on the graphic-text control identification, and control a processing process or a multimedia device participating in the processing process based on the target auxiliary control instruction.

[0041] In an exemplary embodiment of the present disclosure, the system further includes a multimedia device configured to generate multimedia information based on a processing result, and perform a multimedia operation based on the multimedia information.

[0042] According to an aspect of the present disclosure, a written document processing system is provided, which includes: an identification object configured to display a graphic-text control identification; an imaging device configured to acquire the graphic-text control identification from a graphic-text control identification area, and / or acquire the graphic-text control identification from a graphic-text control auxiliary area, and / or acquire a document image of a written document from an interactive object area; a processing device configured to determine a target interactive instruction based on the graphic-text control identification, and process the document image based on the target interactive instruction; and / or determine a target auxiliary control instruction based on the graphic-text control identification, and control a processing process or a multimedia device participating in the processing process based on the target auxiliary control instruction.

[0043] In an exemplary embodiment of the present disclosure, the system further includes a multimedia device configured to generate multimedia information based on a processing result, and perform a multimedia operation based on the multimedia information.

[0044] According to an aspect of the present disclosure, an electronic device is provided, which includes: one or more processors; a storage device configured to store one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0045] According to an aspect of the present disclosure, a computer readable medium is provided, which stores a computer program, and when the program is executed by a processor, the method as described above is implemented.

[0046] It should be understood that the general description above and the following detailed description are only exemplary and are not limiting on the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0047] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout. The following detailed description is presented primarily for the purpose of enabling a person skilled in the art to make and use the disclosure and the best mode and is not intended to limit the scope of the disclosure. The following description is presented along with the accompanying drawings, which are intended to further enable those skilled in the art to make and use the disclosure.

[0048] Figure 1 is a system block diagram of a written document processing method and apparatus according to an exemplary embodiment.

[0049] Figure 2 is a flowchart of a written document processing method according to an exemplary embodiment.

[0050] Figure 3 is a schematic diagram of a written document processing method according to an exemplary embodiment.

[0051] Figure 4 is a flowchart of a written document processing method according to another exemplary embodiment.

[0052] Figure 5 is a schematic diagram of a written document processing method according to an exemplary embodiment.

[0053] Figure 6 is a flowchart of a written document processing method according to another exemplary embodiment.

[0054] Figure 7 is a flowchart of a written document processing method according to another exemplary embodiment.

[0055] Figure 8 is a block diagram of a written document processing apparatus according to an exemplary embodiment.

[0056] Figure 9 is a block diagram of a written document processing apparatus according to another exemplary embodiment.

[0057] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment.

[0058] Figure 11 is a block diagram of a computer readable medium according to an exemplary embodiment. DETAILED DESCRIPTION

[0059] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, however, can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the several views and, thus, description of the same elements will not be repeated.

[0060] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the

[0061] The block diagrams in the drawings show only the functionality of the example embodiments and do not imply any particular physical or architectural arrangement of the example embodiments. No inference should be made regarding the architecture or configuration of a device implementing functionality indicated by any of the diagrams based on an example shown and described herein. Also, well-known structures, devices, and operations have not been shown or described in detail in order to avoid obscuring the disclosure.

[0062] The flow diagrams depicted herein are merely examples of possible flow diagrams and do not necessarily include all of the steps or operations set forth therein. For example, one or more steps or operations can be combined or further partitioned into additional steps or operations. Furthermore, in some embodiments, the steps or operations can be performed in an order different from that shown or described herein.

[0063] It should be understood that, although terms such as first, second, third, etc. can be used herein to describe various components, these components should not be limited by these terms. These terms are used only to distinguish one component from another. Thus, a first component discussed below could be termed a second component without departing from the teachings of the present disclosure. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0064] Those skilled in the art will understand that the modules or flows in the drawings are only schematic diagrams of example embodiments, and are not necessarily essential for implementing the present disclosure, and thus should not be used to limit the protection scope of the present disclosure.

[0065] Figure 1 is a system block diagram of a written document processing method, apparatus, electronic device, and computer readable medium according to an example embodiment.

[0066] As Figure 1As shown, the system architecture 10 can include an identification object 101, an imaging device 102, a network 103 and a processing device 104. The system architecture 10 can also include several multimedia devices 105. The network 103 is a medium to provide a communication link between the imaging device 102 and the processing device 104; the network 103 is also a medium to provide a communication link between the multimedia devices 105 and the processing device 104. The network 104 can include various connection types, such as wired, wireless communication link or fiber cable, etc.

[0067] The identification object 101 is used to display graphic control identification. The identification object 101 can be one-dimensional and / or two-dimensional and / or three-dimensional, and can be selected from one or more of the following: a point, a line, a hatching, a background color, a text, a symbol, a number, a two-dimensional code, a barcode, a picture, a figure, a chart, a two-dimensional or three-dimensional identification object, or other graphic information that can be recognized by computer vision to constitute the identification object 101.

[0068] Furthermore, one and / or multiple identification objects 101 can be combined according to certain rules in time and / or space to form new identification objects 101. When combining the identification objects 101, only the original identification objects 101 can be used, or identification objects 101 for representing the combination relationship between the identification objects 101 can be added to the original identification objects 101.

[0069] The different positions of any one of the identification objects 101, and / or the combination of multiple identification objects 101, and / or the different positions and / or relative positions between multiple identification objects 101, and / or the relative positions between one or more identification objects 101 and the imaging device 102, and / or the different order of appearance and / or disappearance of multiple identification objects 101, and / or the different motion trajectories and / or speeds and / or accelerations and / or angular velocities and / or angular accelerations and / or relative position changes and / or relative speeds and / or relative accelerations and / or relative angular velocities and / or relative angular accelerations and / or vibration frequencies and / or vibration amplitudes of one or more identification objects 101, and / or the color change and / or deformation of one or more identification objects 101, can correspond to different interactive instructions or auxiliary control instructions.

[0070] The graphic control identification can be displayed in a predetermined area by one or more of the following methods: writing, drawing, printing, printing, pasting, displaying, placing, inserting, or other methods that can display the identification object 101, and can be set on the written document or in the appropriate position around it.

[0071] The mapping rule or storage address of the object 101 and the interaction instruction or auxiliary control instruction, the program or storage address for performing the identification and graphic control identification task, and the program or storage address for converting the interaction instruction can be stored on a memory of a client or a server or a cloud, or can be stored in a visual recognition information storage mode (such as a two-dimensional code) on the graphic control identification. When the visual recognition information storage mode (such as a two-dimensional code) is used on the graphic control identification, the content stored in the visual recognition information storage mode on the graphic control identification can be obtained by identifying the graphic control identification.

[0072] The imaging device 102 is configured to acquire the graphic control identification in a preset area and / or acquire a document image of a written document in a target interaction object area. The imaging device 102 interacts with the processing device 104 through the network 103 to receive or send messages and the like. The imaging device 102 can be a smart camera, a camera, a monitoring device, or the like, which includes a camera function.

[0073] The processing device 104 is configured to determine a target interaction instruction and the target interaction object area based on the graphic control identification, process the document image based on the target interaction instruction, determine a target auxiliary control instruction based on the graphic control identification, and control the processing process or a multimedia device participating in the processing process based on the target auxiliary control instruction. The processing device 104 can be installed with various data transmission applications, such as image recognition applications, education platform applications, multimedia applications, and the like. The processing device 104 can be various electronic devices with a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0074] It is worth mentioning that the processing device 104 can be a physical device, a remote server, or a virtual device in the cloud, and the disclosure is not limited thereto.

[0075] The multimedia device 105 is configured to generate multimedia information based on a processing result and perform a multimedia operation based on the multimedia information. The multimedia device 105 can include various types of settings, such as a smart lamp, a smart sound, a multimedia player, a projector, and the like.

[0076] In one specific use scenario, the written document and the marker object 101 are placed on a plane, which can be a desk or other platform, as a preset area. The imaging device 102 can obtain, for example, the graphic-text control marker in the preset area; the processing device 104 can determine, for example, the target interaction instruction and the target interaction object area based on the graphic-text control marker; the imaging device 102 can obtain, for example, the document image of the written document in the target interaction object area; and the processing device 104 can process, for example, the document image based on the target interaction instruction.

[0077] Before the above processing, the processing device 104 can also generate, for example, an interaction instruction mapping rule based on one or more graphic-text control markers and their corresponding interaction instructions; and / or generate an auxiliary control instruction mapping rule based on one or more graphic-text control markers and their corresponding auxiliary control instructions.

[0078] In the above processing, the marker object 101 can be placed on the desktop, for example, the imaging device 102 can obtain, for example, the graphic-text control marker in the preset area; the processing device 104 can extract, for example, the target auxiliary control instruction in the auxiliary control instruction mapping rule based on the graphic-text control marker; and the processing device 104 can control, for example, the processing process or the multimedia device 105 participating in the processing process based on the target auxiliary control instruction.

[0079] The technical content of this use scenario will be described in detail in Figure 2 , Figure 3 the corresponding embodiments.

[0080] In another specific use scenario, the written document and the marker object 101 are placed on a plane, which can be a desk or other platform, as a preset area. The imaging device 102 can obtain, for example, the graphic-text control marker from the graphic-text control marker area; the imaging device 102 can determine, for example, the target interaction instruction based on the graphic-text control marker; the imaging device 102 can obtain, for example, the document image of the written document from the interaction object area; and the processing device 104 can process, for example, the document image based on the target interaction instruction.

[0081] Before the above processing, the processing device 104 can also generate, for example, at least one interaction instruction mapping rule based on one or more graphic-text control markers, at least one graphic-text control marker area, and their corresponding interaction instructions; and / or generate at least one auxiliary control instruction mapping rule based on one or more graphic-text control markers, at least one graphic-text control auxiliary area, and their corresponding auxiliary control instructions.

[0082] In the above processing, the object 101 can be placed on the desktop, for example, the imaging device 102 can obtain the graphic control identifier from the graphic control auxiliary area, for example, the processing device 104 can extract the target auxiliary control instruction in the at least one auxiliary control instruction mapping rule based on the graphic control identifier and the number of the corresponding graphic control auxiliary area, and control the processing process or the multimedia device 105 participating in the processing process based on the target auxiliary control instruction.

[0083] The technical content of this use scenario will be described in detail in Figure 4 , Figure 5 the corresponding embodiments.

[0084] Figure 2 is a flowchart of a written document processing method according to an exemplary embodiment. The written document processing method 20 is a written document processing method for the scenario as Figure 3 described. In Figure 3 the corresponding schematic diagram, the written document is placed on a flat desk, and the user places the identifier object (graphic control identifier) to be processed in any position on the desk in advance. The written document processing method 20 at least includes steps S202 to S208.

[0085] As shown in Figure 2 , in S202, the graphic control identifier in the preset area is obtained. For example, the object image of the identifier object placed in the preset area within the imaging range of the imaging device is obtained, and the identifier object image of the identifier object is identified to generate the graphic control identifier, including: identifying the object image of the identifier object to generate an identification result; and eliminating invalid identifier objects in the identification result to generate the graphic control identifier.

[0086] In one embodiment, before obtaining the graphic control identifier in the preset area, it further includes: generating an interactive instruction mapping rule based on one or more graphic control identifiers and their corresponding interactive instructions; and / or generating an auxiliary control instruction mapping rule based on one or more graphic control identifiers and their corresponding auxiliary control instructions.

[0087] The interactive instruction mapping rule and the auxiliary control instruction mapping rule can specifically include: setting one or more graphic control identifiers; setting an identification method or an identification program of the graphic control identifier, which can be identified according to all or part of the constituent elements of the graphic control identifier; setting an instruction mapping rule for converting the graphic control identifier into a corresponding operation instruction; and further including: setting a graphic control identifier instruction mapping rule supplement or modification method, which supplements or modifies the operation instruction corresponding to a certain graphic control identifier by identifying voice information or graphic information.

[0088] The content of the interaction instruction and the auxiliary control instruction will be described in detail below.

[0089] In S204, the target interaction instruction and the target interaction object region are determined based on the graphic-text control identifier. For example, the target interaction instruction can be extracted from the interaction instruction mapping rule according to the graphic-text control identifier, and the target interaction object region is determined based on the target interaction instruction.

[0090] The interaction object region refers to a specific region in the image of the written document, and the image in the specific region or the elements in the region that can be visually recognized will participate in the specific interaction operation. The interaction object region can be the entire image captured by the imaging device or one or several local regions. When calibrating the interaction object region, it can be described according to its fixed position in the image captured by the imaging device, or according to its relative position with one or several graphic-text control identifiers in the image captured by the imaging device.

[0091] Further, the interaction object region refers to a range that can be captured by the imaging device, and the interaction operation can be performed on the written document image in the range. The interaction object region can be one or multiple, and can be the entire or a part of the written document. Multiple interaction object regions can be in the same written document or in multiple written documents, and the interaction object region can be the entire image captured by the imaging device or a part of the image.

[0092] In S206, the document image of the written document in the target interaction object region is obtained. The obtained written document image can be a written document or a dynamic or static image of the work process on the written document.

[0093] In S208, the document image is processed based on the target interaction instruction. The processing can include: performing image recognition on the document image based on the target interaction instruction to generate the document information; and performing data processing on the document information.

[0094] The image recognition on the document image based on the target interaction instruction to generate the document information includes: determining a target region based on the target interaction instruction; and performing image recognition on the image in the target region in the document image to generate the document information.

[0095] The processing of the document image based on the target interaction instruction can also include: when the graphic-text control identifier requires input of parameters, inputting parameter information according to a preset manner, and the parameters include at least one of voice data, text data, image data, and gesture data.

[0096] According to the written document processing method disclosed herein, a graphic control identifier within a preset area is obtained; a target interaction instruction and a target interaction object area are determined based on the graphic control identifier; a document image of the written document within the target interaction object area is obtained; and the document image is processed based on the target interaction instruction. This method enables multimedia interaction functions of the user's written document without relying on a display screen, and the user can also call upon computer multimedia resources for interaction.

[0097] It should be clearly understood that this disclosure describes how specific examples are formed and used, but the principles of this disclosure are not limited to any details of these examples. Rather, based on the teachings of this disclosure, these principles can be applied to many other embodiments.

[0098] Figure 4 This is a flowchart illustrating a written document processing method according to an exemplary embodiment. The written document processing method 40 is for... Figure 5 The method for processing written documents in the described scenario. Figure 5 In the corresponding diagram, a written document is placed on a flat desk, and the user places the marked objects (graphic control marks) to be processed in one or more preset areas on the desk beforehand. The written document processing method 40 includes at least steps S402 to S408.

[0099] like Figure 4 As shown, in S402, graphic control identifiers are obtained from the graphic control identifier areas. This includes: determining the number of graphic control identifier areas by system settings; and obtaining graphic control identifiers from all graphic control identifier areas.

[0100] One or more graphic control label areas are defined within the imaging device's field of view. These areas can be the entirety of the image captured by the imaging device, or only one or more of those areas. After defining the graphic control label areas, the graphic control labels are identified within each area according to their corresponding mapping rules. The recognizable graphic control labels can be the same or different across different graphic control label areas. Furthermore, the interactive commands corresponding to the same graphic control label can be the same or different across different graphic control label areas.

[0101] The process of obtaining graphic control labels from the entire graphic control label area includes: obtaining an object image of a label object placed in the entire graphic control label area; and performing image recognition on the object image of the label object to generate the graphic control label.

[0102] In an embodiment, before the graphic control identifier is acquired from the graphic control identifier area, the method further comprises: generating at least one interaction instruction mapping rule based on the one or more graphic control identifiers, the at least one graphic control identifier area and the corresponding interaction instruction thereof; and / or generating at least one auxiliary control instruction mapping rule based on the one or more graphic control identifiers, the at least one graphic control auxiliary area and the corresponding auxiliary control instruction thereof.

[0103] The interaction instruction mapping rule and the auxiliary control instruction mapping rule can specifically include: setting one or more graphic control identifiers; setting an identification method or an identification program of the graphic control identifier, and identifying the graphic control identifier according to all or part of the constituent elements of the graphic control identifier; setting an instruction mapping rule for converting the graphic control identifier into a corresponding operation instruction; and / or setting a graphic control identifier instruction mapping rule supplement or modification method for supplementing or modifying the operation instruction corresponding to a certain graphic control identifier by recognizing voice information or graphic information. When supplementing or modifying the existing graphic data according to the information of the graphic control identifier, one or more graphic control auxiliary areas can be set within the imaging device shooting range, and graphic information can be added or changed in the graphic control auxiliary area.

[0104] In S404, a document image of the written document is acquired from the interaction object area.

[0105] In S406, a target interaction instruction is determined based on the graphic control identifier. The method comprises: determining the target interaction instruction based on the graphic control identifier and the number of the graphic control identifier area corresponding thereto.

[0106] The target interaction instruction is determined based on the graphic control identifier and the number of the graphic control identifier area corresponding thereto, which comprises: determining a target interaction instruction mapping rule from the at least one interaction instruction mapping rule based on the number of the graphic control identifier area; and extracting the target interaction instruction from the target interaction instruction mapping rule according to the graphic control identifier.

[0107] In S408, the document image is processed based on the target interaction instruction. The method comprises: performing image recognition on the document image based on the target interaction instruction to generate the document information; and performing data processing on the document information.

[0108] The document image is processed based on the target interaction instruction, which comprises: determining a target area based on the target interaction instruction; and performing image recognition on the image in the target area in the document image to generate the document information.

[0109] The processing of the document image based on the target interaction instruction can further include: when the graphic-text control identifier requires input of a parameter, inputting parameter information according to a preset manner, the parameter including at least one of voice data, text data, image data, and gesture data.

[0110] In one embodiment, the method further includes: adjusting the area range of the graphic-text control identifier region and / or the interactive object region and / or the graphic-text control auxiliary region; and wherein the graphic-text control identifier region, the interactive object region, and the graphic-text control auxiliary region are located within the imaging range of the imaging device.

[0111] The adjusting of the area range of the graphic-text control identifier region and / or the interactive object region and / or the graphic-text control auxiliary region includes: acquiring an object image of an object for marking the area range, the object being placed within the imaging range; performing image recognition on the object image to generate a graphic-text control identifier for marking the area range; determining a target area range according to the graphic-text control identifier for marking the area range; and adjusting the area range of the interactive object region and / or the graphic-text control identifier region and / or the graphic-text control auxiliary region based on the target area range.

[0112] The graphic-text control identifier and the corresponding interaction instruction, which are executable by a computer or other intelligent device, for performing an interactive operation on all or part of the written document image can include:

[0113] Marking the graphic control mark of the interactive range of the written document and the corresponding interactive instruction; marking the graphic control mark of one or more interactive object areas in the interactive range of the written document and the corresponding interactive instruction; selecting the graphic control mark of one or more interactive object areas in the interactive range of the written document and the corresponding interactive instruction for a certain interactive operation; marking the graphic control mark of the identification range of the graphic control mark and the corresponding interactive instruction; marking the graphic control mark of the interactive instruction corresponding to the mapping rule of the identification method of one or more graphic control marks or the program; marking the graphic control mark of the program for identifying the graphic control mark of the interactive operation of the written document and the corresponding interactive instruction; marking the graphic control mark of the program for converting the identified graphic control mark into an interactive instruction and the corresponding interactive instruction; marking the graphic control mark of the program for executing an interactive instruction and the corresponding interactive instruction; marking the graphic control mark of the relevant parameters or data of the interactive instruction of the interactive operation and the corresponding interactive instruction, the required parameters or data of the interactive operation can come from one or more data sources; marking the graphic control mark of the address of the execution program and / or data source of the interactive operation and the corresponding interactive instruction; marking the graphic control mark of the execution program operation permission and / or data source operation permission of the interactive operation and the corresponding interactive instruction; marking the graphic control mark of the annotation or explanation of the interactive process of the written document or the work on the written document and the corresponding interactive instruction; marking the graphic control mark of the explanation or assignment of the attribute of the identified object and the corresponding interactive instruction, the attribute of the identified object including the size information of the writing medium, and / or all or part of the visually identifiable elements printed or printed on the writing medium, and / or the position of the visually identifiable elements on the medium, and / or the relative position between the visually identifiable elements; marking the graphic control mark of the precise positioning of one or more points on the written document or the periphery of the written document and the corresponding interactive instruction, the graphic control mark of the precise positioning of one or more points on the written document or the periphery of the written document refers to one or a group of graphic control marks set for more accurate positioning of the relative positions between multiple points inside or around the written document, the relative positions between one or more points inside or around the written document and the imaging device, and the relative positions between one or more points inside or around the written document and the projection device; marking the graphic control mark of the correction or deformation of the obtained image of the written document and the corresponding interactive instruction; marking the graphic control mark of the projection feedback display area and the corresponding interactive instruction; marking the graphic control mark of the correction or deformation or movement or inversion or rotation of the projection feedback image and the corresponding interactive instruction;The graphic control mark for controlling the device participating in the interactive operation or the device affecting the interactive surrounding environment and the corresponding interactive instruction; the graphic control mark for adjusting the volume, lighting or projection brightness and the corresponding interactive instruction.

[0114] The foregoing graphic control mark and the corresponding interactive instruction are only for more convenient description of the interactive operation. In actual situations, according to the complexity of the interactive operation, the foregoing various interactive operations can be independent of each other, can be combined in an arbitrary manner, can all exist, or can only have an arbitrary one or several types of interactive operation. The interactive operation and the graphic control mark can have a one-to-one and / or one-to-many and / or many-to-many and / or many-to-one corresponding relationship.

[0115] Figure 6 is a flowchart of a written document processing method according to another exemplary embodiment. Figure 6 The flowchart shows a detailed description of the specific processing of the graphic control mark.

[0116] As Figure 6 shown, in S602, the graphic control mark in the preset area is acquired. It is worth mentioning that, in the use scenario of Figure 4 , the imaging device acquires the graphic control mark from the graphic control auxiliary area.

[0117] In S604, the target auxiliary control instruction is extracted in the auxiliary control instruction mapping rule based on the graphic control mark. It is worth mentioning that, in the use scenario of Figure 4 , the processing device extracts the target auxiliary control instruction in the at least one auxiliary control instruction mapping rule based on the graphic control mark and the number of the graphic control auxiliary area corresponding to the graphic control mark.

[0118] In S606, the processing process or the multimedia device participating in the processing process is controlled based on the target auxiliary control instruction.

[0119] When the graphic control mark needs an input parameter, the parameter information can be input according to a preset manner. The parameter includes at least one of voice data, text data, image data and gesture data.

[0120] The recognized interactive instruction can be determined according to the preset graphic control mark and auxiliary control instruction. In the recognition process of the graphic control mark, it can be judged whether the interactive instruction corresponding to a certain graphic control mark needs to be supplemented or changed by recognizing voice information or graphic information. If so, the voice information or graphic information is collected and recognized, and the interactive instruction is supplemented or modified.

[0121] It can also be determined whether the graphic control mark or its corresponding interaction has an interactive object area. If there is an interactive object area, the image of the interactive object area or the elements in the interactive object area that can be visually recognized are obtained as the interactive object of the operation instruction and / or instruction set.

[0122] Figure 7 is a flowchart of a written document processing method according to another exemplary embodiment. Figure 7 The flowchart shows the description of the subsequent processing steps of the document processing.

[0123] As shown in Figure 7 In S702, the multimedia information is generated based on the processing result.

[0124] In S704, the multimedia operation is performed based on the multimedia information.

[0125] In S706, when the multimedia operation is a projection operation, the display area is determined based on the graphic control mark.

[0126] In S708, the document image and / or the projection image projected on the display area are corrected.

[0127] In one embodiment, the first relative position of a known point on the written document corresponding to the document image can be obtained; the second relative position of the known point in the document image can be obtained; the distortion correction mapping between the written document and the document image can be determined based on the first relative position and the second relative position; and the document image can be corrected based on the distortion correction mapping. Specifically, for example, the relative positions between multiple points inside or around the written document and / or the relative positions between one or more points inside or around the written document and the imaging device can be obtained; the images of one or more points inside or around the written document can be obtained; and the distortion correction mapping of one or more points inside or around the written document can be obtained according to the images and the relative positions. The distortion correction of the image makes the image of one or more points inside or around the written document match the relative positions between multiple points inside or around the written document and / or the relative positions between one or more points inside or around the written document and the imaging device.

[0128] In one embodiment, the subsequent processing step of the document processing further comprises: generating an overlay image by overlaying the projected image on the document image after rectification; determining a deviation adjustment mapping of the projected image according to the overlay image; rectifying the projected image based on the deviation adjustment mapping. The feedback image or the calibration image and the expected relative position of the one or more points inside or around the written document with known accurate positioning on the overlay image are obtained. Specifically, for example, the projection of the feedback image or the calibration image, the overlay image of the projection of the feedback image or the calibration image and the image of the one or more points inside or around the written document with known accurate positioning; determining a deviation adjustment mapping of the feedback image or the calibration image according to the actual relative position of the projection of the feedback image or the calibration image and the image of the one or more points inside or around the written document with known accurate positioning on the overlay image; adjusting the feedback image according to the deviation adjustment mapping so that the projection of the feedback image is at the expected relative position of the one or more points inside or around the written document with known accurate positioning on the overlay image.

[0129] Further, the positioning result of the graphic control mark for the accurate positioning of the one or more points on the written document or around the written document is used to position by using the auxiliary line or the auxiliary graphic on the written document display medium, and specifically comprises: obtaining the accurate positioning result of the one or more points on the written document or around the written document; obtaining the auxiliary line or the auxiliary graphic on the written document display medium; obtaining the relative position of the one or more points on the auxiliary line or the auxiliary graphic on the written document display medium and the one or more points with the known accurate positioning result on the written document or around the written document.

[0130] In S710, the processing result is projected on the display area.

[0131] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, the above-mentioned functions defined by the above-mentioned method provided by the present disclosure are executed. The program can be stored in a computer readable storage medium, which can be a read-only memory, a disk or an optical disk, etc.

[0132] In addition, it should be noted that the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0133] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is an apparatus embodiment of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0134] The following is anFigure 8 is a block diagram of a written document processing apparatus according to an example embodiment. As shown in Figure 8 , the written document processing apparatus 80 can be applied in the scenario as described above, comprising an identification object 802, an imaging device 804, a processing device 806, and the written document processing apparatus 80 can further comprise a multimedia device 808. Figure 2

[0135] The identification object 802 is configured to display a graphic-text control identifier.

[0136] The imaging device 804 is configured to acquire the graphic-text control identifier in a preset region, and / or acquire a document image of a written document in a target interactive object region.

[0137] The processing device 806 is configured to determine a target interactive instruction and the target interactive object region based on the graphic-text control identifier, and process the document image based on the target interactive instruction; and / or determine a target auxiliary control instruction based on the graphic-text control identifier, and control a processing process or a multimedia device participating in the processing process based on the target auxiliary control instruction.

[0138] The multimedia device 808 is configured to generate multimedia information based on a processing result, and perform a multimedia operation based on the multimedia information.

[0139] Figure 9 is a block diagram of a written document processing apparatus according to another example embodiment. As shown in Figure 9 , the written document processing apparatus 90 can be applied in the scenario as described above, comprising an identification object 902, an imaging device 904, a processing device 906, and the written document processing apparatus 90 can further comprise a multimedia device 908. Figure 4

[0140] The identification object 902 is configured to display a graphic-text control identifier.

[0141] The imaging device 904 is configured to acquire the graphic-text control identifier from a graphic-text control identifier region, and / or acquire the graphic-text control identifier from a graphic-text control auxiliary region, and / or acquire a document image of a written document from an interactive object region.

[0142] The processing device 906 is configured to determine a target interactive instruction based on the graphic-text control identifier and a number of a graphic-text control identifier region corresponding to the graphic-text control identifier, and process the document image based on the target interactive instruction; and / or determine a target auxiliary control instruction based on the graphic-text control identifier and a number of a graphic-text control auxiliary region corresponding to the graphic-text control identifier, and control a processing process or a multimedia device participating in the processing process based on the target auxiliary control instruction.

[0143] ​​The multimedia device 908 is configured to generate multimedia information based on the processing result, and perform a multimedia operation based on the multimedia information.

[0144] According to the written document processing device of the present disclosure, the image-text control identifier in the preset area is acquired; the target interaction instruction and the target interaction object area are determined based on the image-text control identifier; the document image of the written document in the target interaction object area is acquired; and the document image is processed based on the target interaction instruction. The multimedia interaction function of the written document of the user can be realized without relying on the display screen, and the user can also call the computer multimedia resources for interaction.

[0145] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment.

[0146] The electronic device 1000 according to this embodiment of the present disclosure will be described below with reference to Figure 10 Figure 10 The displayed electronic device 1000 is only an example and should not limit the function and use range of the embodiments of the present disclosure.

[0147] As shown in Figure 10 , the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 can include, but are not limited to, at least one processing unit 1010, at least one storage unit 1020, a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010), a display unit 1040, etc.

[0148] The storage unit stores program codes that can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps described in the present specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 1010 can perform the steps as shown in Figure 2 , Figure 4 , Figure 6 , Figure 7 .

[0149] The storage unit 1020 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 10201 and / or a cache memory 10202, and can further include a read-only memory (ROM) 10203.

[0150] ​The storage unit 1020 can also include a program / utility 10204 having a set (at least one) of program modules 10205, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof, can include implementation of a network environment.

[0151] The bus 1030 can represent one or more of several types of bus structures, including a storage unit bus or bus controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0152] The electronic device 1000 can also communicate with one or more external devices 1000' such as a keyboard, a pointing device, a Bluetooth device, etc. so that a user can interact with the electronic device 1000, and / or any devices (e.g., a router, a modem, etc.) that enable the electronic device 1000 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface 1050. Still yet, the electronic device 1000 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 1060. The network adapter 1060 can be any of a plurality of different types of network adapters known in the art, such as a cable modem, a Digital Subscriber Line (DSL) modem, an Integrated Services Digital Network (ISDN) adapter, a token ring adapter, a wireless network adapter, etc. The network adapter 1060 can include an antenna for transmitting and receiving wireless signals. The network adapter 1060 can be connected to the other components of the electronic device 1000 through the bus 1030. It should be appreciated that the network adapter 1060 can be externally coupled to the electronic device 1000, such as through a network adapter and / or communications portal that is coupled to the bus 1030.

[0153] As those skilled in the art will readily appreciate, the example embodiments described herein can be implemented in a software application, firmware application, circuit application, and / or combination thereof. It should be further appreciated that the software can be implemented in a software package, cartridge, or other software configuration for installation on an electronic device, such as the electronic device 1000. It should be further appreciated that the software can be tangibly embodied, for example, in a machine-readable storage medium or memory having stored thereon instructions that, when executed by a machine, for example, the electronic device 1000, cause the machine to perform methods described herein. Figure 11 As shown, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to perform the above-mentioned method according to the embodiments of the present disclosure.

[0154] The software product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] The computer-readable storage medium can include data signals on a carrier wave modulated or otherwise propagated on a propagation medium. The propagation medium can be any medium that can carry and propagate computer program code, including, but not limited to, electromagnetic, optical, or any suitable combination of the foregoing. The computer-readable storage medium can also be any medium that can be used to store the desired program code, including, but not limited to, a floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] Program code implementing the operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0157] The computer readable medium carries one or more programs, when the one or more programs are executed by the device, the computer readable medium enables the following functions: acquiring a graphic-text control identifier in a preset area; determining a target interaction instruction and a target interaction object area based on the graphic-text control identifier; acquiring a document image of a written document in the target interaction object area; and processing the document image based on the target interaction instruction. The computer readable medium can also enable the following functions: acquiring a graphic-text control identifier from a graphic-text control identifier area; acquiring a document image of a written document from an interaction object area; determining a target interaction instruction based on the graphic-text control identifier; and processing the document image based on the target interaction instruction.

[0158] Those skilled in the art can understand that the above modules can be distributed in the device according to the description of the embodiments, and can also be changed in one or more devices different from the embodiments. The modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0159] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the disclosure.

[0160] The example embodiments of the disclosure are specifically shown and described above. It should be understood that the disclosure is not limited to the detailed structure, arrangement or implementation method described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements within the spirit and scope of the appended claims.

Claims

1. A method of processing a written document, characterized by, The method comprises: acquiring an object image of an identification object placed in a preset area, the imaging range of an imaging device having one or more graphic-text control identification areas and an interactive object area; acquiring a graphic-text control identification in the graphic-text control identification area; determining a target interactive instruction based on the graphic-text control identification and the number of the corresponding graphic-text control identification area; acquiring a document image of a written document in the interactive object area; processing the document image based on the target interactive instruction.

2. The method of claim 1, wherein, The method further comprises: generating at least one interactive instruction mapping rule based on one or more graphic-text control identifications, at least one graphic-text control identification area and the corresponding interactive instruction.

3. The method of claim 1, wherein, The method of acquiring a graphic-text control identification in a graphic-text control identification area comprises: acquiring an object image of an identification object placed in a graphic-text control identification area, the graphic-text control identification area being located in the imaging range of an imaging device; performing image recognition on the object image of the identification object to generate the graphic-text control identification.

4. The method of claim 1, wherein, The method of determining a target interactive instruction and a target interactive object area in an interactive instruction mapping rule based on a graphic-text control identification in a graphic-text control identification area comprises: extracting a target interactive instruction in the interactive instruction mapping rule according to the graphic-text control identification; determining the target interactive object area based on the target interactive instruction.

5. The method of claim 1, wherein, The method of acquiring a graphic-text control identification in a graphic-text control identification area comprises: acquiring a graphic-text control identification from all graphic-text control identification areas.

6. The method of claim 1, wherein, The method further comprises: acquiring an object image of an identification object placed in an imaging range for marking a range of areas, the imaging range having a graphic-text control identification area and an interactive object area; performing image recognition on the object image to generate a graphic-text control identification for marking a range of areas; determining a target range of areas according to the graphic-text control identification for marking a range of areas; adjusting the range of areas of the interactive object area and / or the graphic-text control identification area based on the target range of areas.

7. A system for processing written documents, characterized by The method comprises: an identification object for displaying a graphic-text control identification; an imaging device for acquiring a graphic-text control identification from a graphic-text control identification area, and / or acquiring a graphic-text control identification from a graphic-text control auxiliary area, and / or acquiring a document image of a written document from an interactive object area; a processing device for acquiring an object image of an identification object placed in a preset area, the imaging range of an imaging device having one or more graphic-text control identification areas and an interactive object area; acquiring a graphic-text control identification in the graphic-text control identification area; determining a target interactive instruction based on the graphic-text control identification and the number of the corresponding graphic-text control identification area; acquiring a document image of a written document in the target interactive object area; processing the document image based on the target interactive instruction.

8. The system of claim 7, wherein, The method further comprises: a multimedia device for generating multimedia information based on the processing result; and performing a multimedia operation based on the multimedia information. The method comprises:

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-6. ​ 10. A computer readable medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the method of any one of claims 1-6.

Images