Image information processing method, device and storage medium

Abstract

The present application discloses an image information processing method and device. First, multiple frames of images are acquired, each of which includes at least two frames of images to be processed. Then, a target object is detected in the images to be processed. Based on the detection results of the target object, the sub-handwriting of the handwriting of the multiple frames of images in the images to be processed is determined. Based on the sub-handwriting, the images to be processed are rendered to obtain a target image. The target image includes a writing tool for showing the handwriting. Finally, the target image is sent. In this way, the sub-handwriting of the handwriting of the multiple frames of images in the images to be processed is quickly and accurately determined. The target image is rendered based on the sub-handwriting in the images to be processed to obtain the target image, and the target image is sent, significantly improving the real-time and realistic experience of the handwriting for the user receiving the target image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image information processing method, device, and computer-readable storage medium. Background Art

[0002] With the rapid development of internet technology, video conferencing, online courses, video calls, and other forms of interactive video applications are becoming increasingly widespread. During these interactive video sessions, users often write or draw content to communicate and discuss with other users. Hands and other parts of the body can easily obscure the written content, significantly impacting the viewing experience of the other end of the video. Furthermore, handwritten text cannot be accurately recognized using a single image alone. Summary of the Invention

[0003] The embodiments of the present application provide an image information processing method and apparatus.

[0004] According to a first aspect of the present application, a method for processing image information is provided, the method comprising: acquiring multiple frames of images, the multiple frames of images including at least two frames of images to be processed; detecting a target object in the images to be processed; determining, based on the detection result of the target object, sub-handwriting of the handwriting of the multiple frames of images in the images to be processed; rendering the images to be processed based on the sub-handwriting to obtain a target image, the target image including a writing tool for showing the handwriting; and sending the target image.

[0005] According to an embodiment of the present application, before detecting the target object in the image to be processed, the method further includes: detecting an image rendering instruction, where the image rendering instruction is used to indicate that the image to be processed needs to be rendered according to the sub-handwriting.

[0006] According to one embodiment of the present application, the detection result includes the object position of the target object in the image to be processed; accordingly, determining the sub-handwriting of the writing handwriting of the multiple frames of images in the image to be processed based on the detection result of the target object includes: determining the first position of the sub-object of the target object in the image to be processed based on the object position; and determining the sub-handwriting based on the first position.

[0007] According to one embodiment of the present application, determining the first position of the sub-object of the target object in the image to be processed based on the object position includes: performing target detection on the sub-object to determine the occlusion relationship of the target object to the sub-object; and determining the first position of the sub-object of the target object in the image to be processed based on the object position and the occlusion relationship.

[0008] According to an embodiment of the present application, rendering the image to be processed according to the sub-handwriting to obtain the target image includes: adjusting the transparency attribute of the target object to a transparent value.

[0009] According to an embodiment of the present application, rendering the image to be processed according to the sub-handwriting to obtain a target image further includes: rendering the sub-handwriting in the image to be processed at the object position using a set handwriting style.

[0010] According to an embodiment of the present application, the method further includes: receiving a stop rendering instruction; and stopping rendering the image to be processed according to the sub-handwriting.

[0011] According to an embodiment of the present application, the method further includes: detecting the last frame image of the multiple frames of images; performing image recognition on the last frame image to determine the writing content formed by the handwriting.

[0012] According to an embodiment of the present application, the method further includes: performing semantic recognition on the written content; and optimizing the written content according to the semantic recognition result to obtain optimized target content.

[0013] According to the second aspect of the present application, an image information processing device is also provided, which includes: an acquisition module for acquiring multiple frames of images, wherein the multiple frames of images include at least two frames of images to be processed; a detection module for detecting a target object in the image to be processed; a handwriting determination module for determining a sub-handwriting of the handwriting of the multiple frames of images in the image to be processed based on the detection result of the target object; an image rendering module for rendering the image to be processed based on the sub-handwriting to obtain a target image, wherein the target image includes a writing tool for showing the handwriting; and a sending module for sending the target image.

[0014] The image information processing method and apparatus of the embodiment of the present application first acquires multiple frames of images, each of which includes at least two frames of images to be processed. The method then detects a target object in the images to be processed. Based on the target object detection results, the method determines the sub-handwriting of the handwriting of the multiple frames in the images to be processed. Based on the sub-handwriting, the images to be processed are rendered to obtain a target image, which includes a writing tool used to form the handwriting. Finally, the method transmits the target image. This method allows for the rapid and accurate determination of the sub-handwriting of the handwriting of the multiple frames in the images to be processed, the rendering of the images to be processed to obtain the target image based on the sub-handwriting in the images to be processed, and the transmission of the target image, significantly improving the real-time, authentic experience of the handwriting for the user receiving the target image.

[0015] It should be understood that the teachings of this application do not necessarily achieve all of the beneficial effects described above, but that specific technical solutions can achieve specific technical effects, and other embodiments of this application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0017] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0018] Figure 1 A schematic diagram showing an application scenario of the image information processing method according to an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of the implementation flow of the image information processing method according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram showing multiple frames of images obtained by capturing images of a writing area in the image information processing method according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram showing a target image obtained after rendering multiple frames of captured images using the image information processing method according to an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram showing an implementation flow of an image information processing method according to another embodiment of the present application is shown;

[0023] Figure 6 A schematic diagram showing the structure of an image information processing device according to an embodiment of the present application is shown;

[0024] Figure 7 A schematic diagram of the composition structure of the device in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0026] The technical solution of the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Figure 1 A schematic diagram of an application scenario of the image information processing method according to an embodiment of the present application is shown.

[0028] refer to Figure 1 The image information processing method of the embodiment of the present application can be applied to video interaction scenarios such as video teaching and video conferencing. The video interaction process in the video interaction scenario can be an end-to-end video interaction process or a distributed video interaction process. The video interaction system used in the video interaction process includes at least two clients, client 11 and client 12, and can also include a central server 13. Clients 11 and 12, client 11 and central server 13, and client 12 and central server 13 are all in communication connection.

[0029] In the embodiment of the present application, the image information processing method provided by the embodiment of the present application can be described by taking the client 11 participating in the video interaction as an example. Figure 1 As shown, the image acquisition device 101 of the client 11 can capture images of a writing area 102 where the client user performs writing operations, thereby obtaining multiple frames of images. It should be noted that the writing area 102 can be an area of ​​a display device 103 configured on the client for writing operations, or it can be an area where the client user uses a common whiteboard, paper, or the like and a writing pen to perform writing operations. Here, the connection between the image acquisition device 101 and the display device 103 can be wired or wireless.

[0030] It should be noted that the image information processing method of the present application can also be applied to the central server of a distributed video interaction system, which may include multiple clients and at least one central server. The central server obtains multiple frames of images from one of the clients, processes the image information of the image to be processed in the multiple frames using the information processing method of the present application, obtains the target image, and then sends the target image to all clients, or only to a specified client.

[0031] Figure 2 A schematic diagram of the implementation flow of the image information processing method according to an embodiment of the present application is shown.

[0032] refer to Figure 2The image information processing method of the embodiment of the present application includes at least the following operation flow: operation 201, acquiring multiple frames of images, wherein the multiple frames of images include at least two frames of images to be processed; operation 202, detecting the target object in the image to be processed; operation 203, determining the sub-handwriting of the handwriting of the multiple frames of images in the image to be processed based on the detection result of the target object; operation 204, rendering the image to be processed based on the sub-handwriting to obtain a target image, wherein the target image includes a writing tool for showing the formation of the handwriting; operation 205, sending the target image.

[0033] In operation 201 , multiple frames of images are acquired, where the multiple frames of images include at least two frames of images to be processed.

[0034] In this embodiment of the present application, the multiple frames of images may be multiple images in video data in application scenarios such as video conferencing, video calls, and video teaching. Taking the video conferencing application scenario as an example, during a video conference, a user writes on a piece of paper, a whiteboard, or a blackboard, and the image acquisition device can capture multiple frames of images, which may include images that can show the user's writing process.

[0035] Figure 3 A schematic diagram of multiple frames of images obtained by capturing images of a writing area in the image information processing method according to an embodiment of the present application is shown.

[0036] refer to Figure 3 The multiple frames of images acquired in the image information processing method of the embodiment of the present application include m frames of images. In the embodiment of the present application, the images to be processed can be the nth and n+1th frames among the m frames of images, or can be the m-1th and mth frames, the last two frames of the m frames of images.

[0037] In operation 202 , a target object is detected in an image to be processed.

[0038] In this embodiment of the present application, the target object can be the hand of the user performing the writing operation or the writing pen used for writing, or the user's hand or the writing pen can be used as the target object at the same time. Here, the target object can be detected using a common target detection technology in the field of image processing.

[0039] In this embodiment of the present application, the detection result of the target object may include: the object position of the target object in the image to be processed.

[0040] Return Reference Figure 3, taking the images to be processed as the nth frame and the n+1th frame as an example, in the process of processing the information of the nth frame image, the client user's hand and the writing pen can be used as target objects at the same time, and the object position of the sub-image of the user holding the writing pen in the writing area can be detected. Here, the image coordinate system can be configured in advance according to the shape and size of the writing area, and the center point of the writing area can be used as the coordinate origin, the horizontal direction as the horizontal coordinate axis, and the vertical direction as the vertical coordinate axis. The lower left side of the writing area can also be used as the coordinate origin, the horizontal direction as the horizontal coordinate, the right as the positive direction of the horizontal coordinate, the vertical direction as the vertical coordinate axis, and the upward direction as the positive direction of the vertical coordinate. After target detection is performed on the image to be processed, the outline of the target object and the coordinate information of each point on the target object in the image coordinate system are obtained as the object position of the target object.

[0041] In this embodiment of the present application, the detection result of the target object may further include: whether the hand blocks the writing pen, whether the hand or the writing pen blocks the handwriting, etc. In addition, whether other parts of the user's body block the writing pen or the handwriting may also be detected at the same time.

[0042] Return for further reference Figure 3 When the target object is detected, the sub-objects of the target object can be further detected. Here, the user's hand and the writing pen can be detected first, and then the pen tip can be detected. If the hand and the writing pen are detected, but the pen tip is not detected, it can be determined that the hand and the pen tip form an occlusion, and you can refer to Figure 3 The mth frame image in .

[0043] In this embodiment of the present application, before detecting the target object in the image to be processed, an image rendering instruction is detected, where the image rendering instruction is used to indicate that the image to be processed needs to be rendered according to the sub-handwriting.

[0044] In this embodiment of the present application, the image rendering command can be configured based on conference attributes such as the importance level of the video conference and the number of participants. The image rendering command is automatically triggered when the conference attributes meet the configured conditions. It can also be configured as a physical or virtual button, set as a user-selectable option, to issue the image rendering command when the user desires it.

[0045] Return Reference Figure 1 In this embodiment of the present application, the image rendering instruction virtual key used to represent the need to render the processed image according to the sub-handwriting can be configured on the writing area 102 of the display device 103, and can also be configured on other operation interfaces of the client.

[0046] In operation 203, according to the detection result of the target object, the sub-handwriting of the writing handwriting of multiple frames of images in the image to be processed is determined.

[0047] In this embodiment of the present application, the sub-handwriting can be a point formed by the user's writing or a short line formed by the user's writing, etc.

[0048] Reference Figure 3 In the second frame of the image, compared with the first frame, there is an additional stroke "丶". Then, in the process of image information processing for the second frame of the image, it can be determined that the sub-handwriting of the writing handwriting formed by m frames of images in the second frame of the image is a "丶".

[0049] In this embodiment of the present invention, the multiple frames of images can be continuous or discontinuous multiple frames of images with a determined time sequence in video data in scenarios such as video conferencing and video calls. Since the speed of the user's writing process is fast or slow and the writing content is not a fixed line, the length and line shape of the sub-handwriting of each frame of the multiple frames of images can be different.

[0050] In this embodiment of the present application, if the detection result includes the object position of the target object in the image to be processed, correspondingly, the first position of the sub-object of the target object in the image to be processed can be determined according to the object position. Further, according to the first position, the sub-handwriting is determined.

[0051] In this embodiment of the present invention, the target object can be a writing pen, and the hand of the user performing the writing operation and the writing pen can also be regarded as the same target object. The sub-object can be the tip of the writing pen.

[0052] The tip of the pen has a determined relative position with the writing pen, and the writing handwriting in multiple frames of images is formed at the tip position. Therefore, by performing target detection on the image to be processed, the positions of the user's hand and the writing pen can be determined, and further, according to the positions of the user's hand and the writing pen, the first position of the tip of the pen can be determined.

[0053] In this embodiment of the present invention, the position of the tip of the pen is the end position of the sub-handwriting in the corresponding image to be processed. And there is a previous frame of the image in the multiple frames of images that is temporally before the image to be processed. The position where the sub-handwriting of the previous frame of the image ends can be used as the start position of the sub-handwriting of the current image to be processed. From the start position to the end position, it can be considered as the sub-handwriting in the current image to be processed. If the displacement from the start position to the end position is particularly small, it can be approximated as a point. If the displacement from the start position to the end position exceeds the first set displacement value, it can be approximated as a line segment. Multiple other sub-handwriting graphics can also be configured. When the displacement from the start position to the end position is the second set displacement value, according to the partial writing handwriting of the multiple frames of images that have been determined in the image to be processed, the graphics of the sub-handwriting of the image to be processed are configured.

[0054] Here, the partial handwriting of the multiple frames of images determined in the image to be processed refers to the writing content appearing in the image to be processed, but the writing content is formed in other images that are located before the image to be processed in time sequence in the multiple frames of images.

[0055] In this embodiment of the present application, target detection can be performed on the sub-object to determine the occlusion relationship of the target object to the sub-object, and the first position of the sub-object of the target object in the image to be processed can be determined based on the object position and the occlusion relationship.

[0056] In another embodiment of the present application, the detection result also includes the object position of the target object in the image to be processed, but the target object is only the hand of the user performing the writing operation. Accordingly, based on the object position, the second position of the second image having a certain relative position relationship with the target object in the image to be processed can be determined, and further, the sub-handwriting can be determined based on the second position.

[0057] In this embodiment of the present application, if it is detected that the target object occludes the sub-object, the image that is located before the image to be processed in the multi-frame image sequence can be used to predict the predicted position of the sub-handwriting of the handwriting of the multi-frame image in the image to be processed. The operation of predicting the position of the sub-handwriting can adopt the image detection and image prediction technology commonly used in the field. Because, if the target object occludes the sub-object, it is possible that the target object occludes the sub-object in several consecutive frames of images. Therefore, the image that is located before the image to be processed in the multi-frame image sequence can be the previous frame of the image to be processed, or it can be the previous two frames or the previous three frames, etc.

[0058] In operation 104 , the image to be processed is rendered according to the sub-handwriting to obtain a target image, where the target image includes a writing tool used to form the handwriting.

[0059] In this embodiment of the present application, rendering the image to be processed according to the sub-handwriting to obtain the target image may include: adjusting the transparency attribute of the target object to a transparent value.

[0060] In this embodiment of the present application, the transparency attribute of the target object can be attributes such as image brightness, image grayscale value and contrast. The value of at least one of them can be adjusted to make the visual state of the target object transparent or near-transparent. In actual application, it can be set according to actual needs.

[0061] In this embodiment of the present application, a handwriting style may be set to render the sub-handwriting in the image to be processed at the object position, so as to render the image to be processed according to the sub-handwriting and obtain the target image.

[0062] In this embodiment of the present application, the writing tool used to form the handwriting in the target image may be a virtual writing pen. The form of the virtual writing pen may be configured and selected according to requirements.

[0063] In this embodiment of the present application, setting the handwriting style may include the style of the writing pen, the thickness of the handwriting, the color of the handwriting, and the handwriting effect of the handwriting, etc. The handwriting effect of the handwriting may include shadow and filling, etc.

[0064] In this embodiment of the present application, the image information processing during a video conference is taken as an example. When a user writes on paper during a video conference, the user's hand or writing pen that causes the text to be obscured can be made transparent in real time. At the same time, in order to enable the user who receives the image or video data to visually obtain a more realistic viewing experience of the writing process, the user's writing gestures and other information can be determined based on the user's hand and writing pen obtained during the image detection process. Furthermore, based on the writing pen gestures, the writing pen is presented in real time as a virtual transparent state image, and the gestures of the virtual transparent pen are consistent with the gestures of the user's writing pen.

[0065] Figure 4 A schematic diagram shows a target image obtained after rendering multiple frames of captured images using the image information processing method according to an embodiment of the present application.

[0066] refer to Figure 4 After processing the m frames of images in sequence using the image information processing method of the embodiment of the present application, the following can be obtained: Figure 4 The target image after a set of m frames of rendering is shown in . It should be noted here that Figure 4 The rendering effect in the video is merely an example. In actual use, the virtual transparent image of the writing pen can be achieved by adding a layer containing a virtual writing pen. The brightness, contrast, grayscale value, or other transparency properties of this layer can be set. In addition, the style of the writing tool used in the rendering process can be set according to the user's aesthetic preferences. This can preserve the user's actual writing process while effectively preventing the obstruction of the written content by the user's hand or writing pen. This effectively enhances the user's realistic experience of the writing process during video interaction.

[0067] In operation 105 , the target image is transmitted.

[0068] In this embodiment of the present application, the image information processing method of the present application can be applied to an end-to-end video interaction process, and can also be used in a distributed video interaction process.

[0069] In this embodiment of the present application, the process of processing the image to be processed can be as follows: Figure 1 The client 11 shown is performed, and can also be performed on the client 12 that receives the image, or on the central server 13. In actual application, configuration can be performed according to needs, and this application does not limit this.

[0070] In this embodiment of the present application, a stop rendering instruction is also received to stop rendering the image to be processed according to the sub-handwriting.

[0071] In this embodiment of the present application, the stop rendering instruction can be automatically issued upon detecting that the image to be processed is the last frame of the user's handwriting process. Alternatively, the stop rendering instruction can be issued based on the smoothness of video interaction or other practical requirements. Upon receiving the stop rendering instruction, rendering of the image to be processed based on the sub-handwriting is stopped.

[0072] Figure 5 A schematic diagram of the implementation flow of an image information processing method according to another embodiment of the present application is shown.

[0073] refer to Figure 5 The image information processing method of this embodiment of the present application at least includes the following operation procedures:

[0074] Operation 501 is to acquire multiple frames of images, where the multiple frames of images include at least two frames of images to be processed.

[0075] Operation 502: Detect a target object in an image to be processed.

[0076] Operation 503 : determining the sub-handwritings of the handwritings of the multiple frames of images in the image to be processed according to the detection result of the target object.

[0077] In operation 504 , the image to be processed is rendered according to the sub-handwriting to obtain a target image, where the target image includes a writing tool used to form the handwriting.

[0078] Operation 505: Send the target image.

[0079] In operation 506 , the last frame of the multiple frames is detected.

[0080] In this embodiment of the present application, detecting the last frame of the multi-frame image can be achieved by image recognition in the image to be processed. For example, detecting that an image of a set size is full of writing content formed by writing tracks.

[0081] In this embodiment of the present application, the difference between multiple frames can also be used to determine whether the image to be processed is the last frame. If the difference between a set number of consecutive frames is less than the set difference, it can be determined that the user has not performed a writing operation within the set time, and the current image to be processed can be determined to be the last frame of the multiple frames.

[0082] In another embodiment of the present application, the last frame of the multiple frames can be detected based on audio data corresponding to the multiple frames. When the audio data indicates that the user has declared that the current writing operation has ended, the current image to be processed can be determined to be the last frame of the multiple frames.

[0083] Operation 507 : performing image recognition on the last frame image to determine the written content formed by the handwriting.

[0084] In this embodiment of the present application, OCR (Optical Character Recognition) technology can be used to perform image recognition on the last frame image to determine the written content formed by the handwriting. The recognition result can be the conversion of the handwritten text or graphics formed by the writing trajectory into an electronic document.

[0085] In this way, while using the image information processing method provided in the embodiment of the present application to display the writing process, it can also provide users with the written content formed by the handwriting during the writing process, making it convenient for users to record and archive the written content formed by the handwriting during the video interaction process, further improving the user experience.

[0086] Operation 508: Perform semantic recognition on the written content.

[0087] In this embodiment of the present application, semantic recognition of written content can be achieved by using common semantic recognition technologies in the field such as NLP (Natural Language Processing).

[0088] Operation 509 : Optimize the written content according to the semantic recognition result to obtain optimized target content.

[0089] In this embodiment of the present application, after semantic recognition of the written content, obvious typos, graphics, and symbols can be corrected, and the written content formed by the writing trajectory can be efficiently and quickly optimized.

[0090] In this embodiment of the present application, the optimized target content may also be sent to the client.

[0091] In this way, through the above operations, in addition to achieving a realistic presentation of the writing process and a complete display of the user's writing trajectory, the written content formed by the writing trajectory can also be extracted in real time. Furthermore, these continuous multi-frame images with a time-series relationship can be used to perform semantic recognition of the writing trajectory and content in the image, obtaining more accurate recognition results. Furthermore, the written content can be converted into an electronic document, making it easier for users to record the content of the video interaction process later.

[0092] Among them, the specific implementation process of operations 501 to 505 is the same as Figure 2 The specific implementation processes of operations 201 to 205 in the illustrated embodiment are similar and will not be repeated here.

[0093] It should be noted that operations 508 and 509 are operations of a more preferred embodiment. In this embodiment of the present invention, operations 508 and 509 may not be included.

[0094] The image information processing method and apparatus of the embodiment of the present application first acquires multiple frames of images, each of which includes at least two frames of images to be processed. The method then detects a target object in the images to be processed. Based on the target object detection results, the method determines the sub-handwriting of the handwriting of the multiple frames in the images to be processed. Based on the sub-handwriting, the images to be processed are rendered to obtain a target image, which includes a writing tool used to form the handwriting. Finally, the method transmits the target image. This method allows for the rapid and accurate determination of the sub-handwriting of the handwriting of the multiple frames in the images to be processed, the rendering of the images to be processed to obtain the target image based on the sub-handwriting in the images to be processed, and the transmission of the target image, significantly improving the real-time, authentic experience of the handwriting for the user receiving the target image.

[0095] Similarly, based on the above image information processing method, an embodiment of the present application also provides a computer-readable storage medium, which stores a program. When the program is executed by a processor, the processor performs at least the following operation steps: operation 201, obtaining multiple frames of images, the multiple frames of images including at least two frames of images to be processed; operation 202, detecting the target object in the image to be processed; operation 203, determining the sub-handwriting of the handwriting of the multiple frames of images in the image to be processed based on the detection result of the target object; operation 204, rendering the image to be processed based on the sub-handwriting to obtain a target image, the target image including a writing tool for showing the formation of the handwriting; operation 205, sending the target image.

[0096] Furthermore, based on the above image information processing method, the embodiment of the present application also provides an image information processing device, such as Figure 6The device 60 includes: an acquisition module 601, used to acquire multiple frames of images, wherein the multiple frames of images include at least two frames of images to be processed; a detection module 602, used to detect a target object in the image to be processed; a handwriting determination module 603, used to determine the sub-handwriting of the handwriting of the multiple frames of images in the image to be processed according to the detection result of the target object; an image rendering module 604, used to render the image to be processed according to the sub-handwriting to obtain a target image, wherein the target image includes a writing tool for showing the handwriting; and a sending module 605, used to send the target image.

[0097] In this embodiment of the present application, the device 60 further includes: an instruction detection module, configured to detect an image rendering instruction before detecting a target object in the image to be processed, wherein the image rendering instruction is used to indicate that the image to be processed needs to be rendered according to the sub-handwriting.

[0098] In this embodiment of the present application, the detection result includes the object position of the target object in the image to be processed; accordingly, the handwriting determination module 603 includes: a position determination submodule, used to determine the first position of the sub-object of the target object in the image to be processed based on the object position; and a handwriting determination submodule, used to determine the sub-handwriting based on the first position.

[0099] In this embodiment of the present application, the position determination submodule includes: an occlusion determination unit, which is used to perform target detection on the sub-object and determine the occlusion relationship of the target object to the sub-object; and a position determination unit, which is used to determine the first position of the sub-object of the target object in the image to be processed based on the object position and the occlusion relationship.

[0100] In this embodiment of the present application, the image rendering module includes: a transparency submodule, which is used to adjust the transparency attribute of the target object to a transparent value.

[0101] In this embodiment of the present application, the image rendering module 604 further includes: a rendering submodule, configured to render the sub-handwriting in the image to be processed at the object position using a set handwriting style.

[0102] In this embodiment of the present application, the apparatus 60 further includes: a stop detection module for receiving a stop rendering instruction; and a stop rendering module for stopping rendering the image to be processed according to the sub-handwriting.

[0103] In this embodiment of the present application, the device 60 further includes: an image recognition module for detecting the last frame image of multiple frames; and a content determination module for performing image recognition on the last frame image to determine the writing content formed by the handwriting.

[0104] In this embodiment of the present application, the device 60 further includes: a semantic recognition module for performing semantic recognition on the written content; and a content optimization module for optimizing the written content according to the semantic recognition result to obtain optimized target content.

[0105] Furthermore, based on the above image information processing method, the embodiment of the present application also provides a device, such as Figure 7 As shown, the device 70 includes at least one processor 701, and at least one memory 702 and a bus 703 connected to the processor 701; wherein the processor 701 and the memory 702 communicate with each other through the bus 703; the processor 701 is used to call the program instructions in the memory 702 to execute the above-mentioned image information processing method.

[0106] It should be noted that the above description of the embodiments of the image information processing device and equipment is different from the above description of the embodiments of the image information processing device and equipment. Figures 1 to 5 The description of the method embodiment shown is similar, with the same Figures 1 to 5 For technical details not disclosed in the embodiments of the image information processing device and equipment of this application, please refer to the aforementioned Figures 1 to 5 The description of the method embodiment shown in the figure is understood, and in order to save space, it is not repeated here.

[0107] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0109] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0110] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0111] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0112] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0113] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for processing image information, the method comprising: Acquire multiple frames of images, wherein the multiple frames of images include at least two frames of images to be processed; Detecting a target object in the image to be processed, where the target object is a user's hand and / or a writing pen; Determining, based on the detection result of the target object, sub-handwritings of the handwritings of the multiple frames of images in the image to be processed; Rendering the image to be processed according to the sub-handwriting to obtain a target image, wherein the target image includes a writing tool for showing the handwriting; sending the target image; Rendering the image to be processed according to the sub-handwriting to obtain a target image includes: Adjust the transparency attribute of the target object to a transparent value.

2. The method according to claim 1, before detecting the target object in the image to be processed, the method further comprises: An image rendering instruction is detected, where the image rendering instruction is used to indicate that the image to be processed needs to be rendered according to the sub-handwriting.

3. The method according to claim 1, wherein the detection result includes the object position of the target object in the image to be processed; The step of determining sub-handwritings of the handwritings of the multiple frames of images in the image to be processed according to the detection result of the target object includes: Determining a first position of a sub-object of the target object in the image to be processed according to the object position; The sub-handwriting is determined according to the first position.

4. The method according to claim 3, wherein determining the first position of the sub-object of the target object in the image to be processed according to the object position comprises: Performing target detection on the sub-object to determine an occlusion relationship between the target object and the sub-object; Determine a first position of a sub-object of the target object in the image to be processed according to the object position and the occlusion relationship.

5. The method according to claim 1, wherein the step of rendering the image to be processed according to the sub-handwriting to obtain a target image further comprises: The sub-handwriting in the image to be processed is rendered at the object position using a set handwriting style.

6. The method according to claim 1, further comprising: Receive a stop rendering instruction; Stop rendering the image to be processed according to the sub-handwriting.

7. The method according to claim 1, further comprising: Detecting a last frame image of the multiple frames of images; Image recognition is performed on the last frame image to determine the writing content formed by the writing handwriting.

8. The method according to claim 7, further comprising: performing semantic recognition on the written content; The written content is optimized according to the semantic recognition result to obtain optimized target content.

9. An image information processing device, comprising: An acquisition module, configured to acquire multiple frames of images, wherein the multiple frames of images include at least two frames of images to be processed; a detection module, configured to detect a target object in the image to be processed, wherein the target object is a user's hand and / or a writing pen; a handwriting determination module, configured to determine sub-handwritings of the handwritings of the multiple frames of images in the image to be processed according to the detection result of the target object; An image rendering module, configured to render the image to be processed according to the sub-handwriting to obtain a target image, wherein the target image includes a writing tool used to form the handwriting; A sending module, configured to send the target image; The image rendering module includes: a transparency submodule, which is used to adjust the transparency attribute of the target object to a transparent value.

Images