Method and system for character recognition in thangka images based on human posture estimation

Abstract

The present invention discloses a thangka image character recognition method and system based on human posture estimation, which is applied to image recognition technology, and the method includes: obtaining foreground image information; forming limb key point information and supplementary recognition information; forming multiple enhanced arm information and multiple enhanced head information; forming multiple groups of head-hand combination information and multiple groups of information to be recognized; inputting multiple groups of information to be recognized corresponding to thangka images to be recognized into a preset recognition model for recognition to obtain multiple groups of recognition results, and obtaining character information corresponding to the thangka images to be recognized according to the multiple groups of recognition results. The present invention provides a thangka image character recognition method and system based on human posture estimation, and provides a technical solution for improving the existing human posture estimation algorithm to realize character recognition in response to the complex human body composition problem of characters in thangka images, which can cope with character recognition in different postures in thangkas, and help us more accurately recognize and understand the identity of characters in thangka art.

Description

Technical Field

[0001] The present invention relates to image recognition technology, and in particular to a thangka image character recognition method and system based on human body posture estimation. Background Art

[0002] Deep learning has made significant progress in the fields of image processing and visual recognition. The image of the human figure in Thangka is particularly important. Although deep learning has made great progress in recognizing and classifying conventional images, its application in recognizing the identity of Thangka figures with complex patterns and symbolic meanings is relatively rare. A major feature of Thangka artworks is that the characters often have many features, which poses new challenges to existing human pose estimation algorithms. Summary of the invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a method and system for thangka image character recognition based on human posture estimation.

[0004] In a first aspect, the present application provides a method for character recognition in thangka images based on human posture estimation, comprising:

[0005] Preprocess the thangka image to be identified to obtain foreground image information containing only the person image;

[0006] Performing human body key point recognition on the foreground image information to form limb key point information, and capturing image features corresponding to different limbs to form supplementary recognition information; the limb key point information includes arm key point information, leg key point information, torso key point information and head key point information;

[0007] According to the positional relationship between the arm key point information and the torso key point information, the arm key point information is increased or decreased to form a plurality of enhanced arm information, and according to the head key point information, the head key point information is increased or decreased to form a plurality of enhanced head information;

[0008] Randomly combining the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and combining the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified;

[0009] Multiple groups of information to be identified corresponding to the thangka image to be identified are input into a preset recognition model for identification to obtain multiple groups of identification results, and the character information corresponding to the thangka image to be identified is obtained according to the multiple groups of identification results.

[0010] When the embodiment of the present application is implemented, in order to reduce the image features in the thangka image, it is necessary to first extract the foreground image information containing only the character image, which can be extracted based on the existing mask technology, and the embodiment of the present application does not limit it. In order to accurately identify the character, the embodiment of the present application adopts a scheme of first performing human body key point recognition, wherein the limb key point information to be identified needs to include arm key point information, leg key point information, torso key point information and head key point information. For example, OpenPose is used to mark and identify the joints, and the identified key points are formed into multiple vectors. It should be understood that after the key point information is identified based on the prior art, the identified limb key point information needs to include the coordinates of the key point, the corresponding part of the key point and the vector formed between different key points. These contents belong to the prior art, and the embodiment of the present application does not repeat them. At the same time, in order to supplement the relevant features, it is necessary to capture the supplementary identification information. Specifically, the supplementary identification information generally corresponds to different limb parts, such as the gesture corresponding to the wrist, the sitting posture corresponding to the leg, and the face corresponding to the head.

[0011] In the embodiments of the present application, the inventors found that the characters in the thangka images may appear in different forms in different images, such as three heads and eight arms in one thangka image, and one head and two arms in another thangka image. Therefore, in order to improve the accuracy of recognition, the embodiments of the present application enhance the acquired arm key point information and head key point information, wherein the enhancement method is that for the arm key point information, it is necessary to refer to the relative position of the arm key point information and the torso key point information for increase or decrease calculation, while the head key point information can be enhanced only based on the position of the head key point information itself. After the enhanced information is combined, multiple groups of combined information can be formed, and after the combined information and the key point information of the legs and torso are combined, information for identification can be formed.

[0012] In the embodiment of the present application, it is necessary to pre-train the recognition model, which can be implemented based on a graph convolutional network. After the recognition model recognizes the information to be recognized, multiple results can be output, and then the characters in the thangka image can be more accurately recognized based on the distribution of these multiple results. The embodiment of the present application provides a technical solution for improving the existing human posture estimation algorithm to achieve character recognition in response to the complex human body composition problem of the characters in the thangka image through the above technical solution. It can cope with the recognition of characters in different postures in thangka, helping us to more accurately identify and understand the identity of the characters in thangka art.

[0013] In a possible implementation, capturing image features corresponding to different limbs to form supplementary identification information includes:

[0014] Capturing image features of the parts corresponding to the key point information of the arms to form gesture features, capturing image features of the parts corresponding to the key point information of the legs to form sitting posture features, and capturing image features of the parts corresponding to the key point information of the head to form facial features;

[0015] The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

[0016] In a possible implementation, adding or reducing the arm key point information to form a plurality of enhanced arm information according to the positional relationship between the arm key point information and the torso key point information includes:

[0017] Acquire the shoulder key point information, the elbow key point information and the wrist key point information in the arm key point information, and calculate the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculate the vector direction between the wrist key point information and the elbow key point information as the lower arm direction;

[0018] When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion;

[0019] When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation;

[0020] New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

[0021] In a possible implementation manner, adding or reducing the head key point information according to the head key point information to form a plurality of enhanced head information includes:

[0022] When there is only one group of head key point information, taking the group as an axis, generating new head key point information of corresponding angles on both sides of the group;

[0023] When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

[0024] In a possible implementation, the generation of the recognition model includes:

[0025] Acquire multiple thangka images and extract character images as a sample image group;

[0026] Extracting key points of a human body for the sample image group, and forming a first sample pair with the extracted key points and the names of corresponding people;

[0027] Extracting image feature information corresponding to different human key points in the sample image group as supplementary sample information, and adding the supplementary sample information to the first sample pair to form a second sample pair; the supplementary sample information includes gesture features, sitting posture features and facial features;

[0028] The recognition model is formed by training a model using the key points and supplementary sample information in the second sample pair as input information and the names of the corresponding characters in the second sample pair as output information.

[0029] In a possible implementation, obtaining the character information corresponding to the thangka image to be identified according to the multiple sets of identification results includes:

[0030] The character information with the highest proportion is selected from the multiple groups of recognition results as the character information corresponding to the thangka image to be recognized.

[0031] In a second aspect, the embodiment of the present application further provides a thangka image character recognition system based on human posture estimation, comprising:

[0032] A preprocessing unit is configured to preprocess the thangka image to be identified to obtain foreground image information containing only the image of the person;

[0033] A key point unit is configured to perform human key point recognition on the foreground image information to form limb key point information, and to capture image features corresponding to different limbs to form supplementary recognition information; the limb key point information includes arm key point information, leg key point information, torso key point information and head key point information;

[0034] an enhancement unit, configured to increase or decrease the arm key point information according to the positional relationship between the arm key point information and the torso key point information to form a plurality of enhanced arm information, and to increase or decrease the head key point information according to the head key point information to form a plurality of enhanced head information;

[0035] a combining unit configured to randomly combine the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and to combine the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified;

[0036] The recognition unit is configured to input multiple groups of information to be recognized corresponding to the thangka image to be recognized into a preset recognition model for recognition to obtain multiple groups of recognition results, and obtain the character information corresponding to the thangka image to be recognized based on the multiple groups of recognition results.

[0037] In a possible implementation, the key point unit is further configured as:

[0038] Capturing image features of the parts corresponding to the key point information of the arms to form gesture features, capturing image features of the parts corresponding to the key point information of the legs to form sitting posture features, and capturing image features of the parts corresponding to the key point information of the head to form facial features;

[0039] The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

[0040] In a possible implementation manner, the enhancement unit is further configured to:

[0041] Acquire the shoulder key point information, the elbow key point information and the wrist key point information in the arm key point information, and calculate the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculate the vector direction between the wrist key point information and the elbow key point information as the lower arm direction;

[0042] When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion;

[0043] When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation;

[0044] New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

[0045] In a possible implementation manner, the enhancement unit is further configured to:

[0046] When there is only one group of head key point information, taking the group as an axis, generating new head key point information of corresponding angles on both sides of the group;

[0047] When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] The present invention provides a thangka image character recognition method and system based on human posture estimation, and provides a technical solution for improving the existing human posture estimation algorithm to achieve character recognition in response to the complex human body composition problem in thangka images. The method and system can cope with character recognition in different postures in thangkas, and help us to more accurately identify and understand the identities of characters in thangka art. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0051] Figure 1 This is a schematic diagram of the method steps of an embodiment of the present application. DETAILED DESCRIPTION

[0052] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0053] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0054] Please refer to Figure 1 , which is a flow chart of a method for character recognition in thangka images based on human posture estimation provided in an embodiment of the present invention. Furthermore, the method for character recognition in thangka images based on human posture estimation may specifically include the contents described in the following steps S1 to S5.

[0055] S1: pre-processing the thangka image to be identified to obtain foreground image information containing only the person image;

[0056] S2: performing human body key point recognition on the foreground image information to form limb key point information, and capturing image features corresponding to different limbs to form supplementary recognition information; the limb key point information includes arm key point information, leg key point information, torso key point information and head key point information;

[0057] S3: adding or reducing the arm key point information according to the positional relationship between the arm key point information and the torso key point information to form a plurality of enhanced arm information, and adding or reducing the head key point information according to the head key point information to form a plurality of enhanced head information;

[0058] S4: randomly combining the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and combining the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified;

[0059] S5: Inputting multiple groups of information to be identified corresponding to the thangka image to be identified into a preset recognition model for identification to obtain multiple groups of identification results, and obtaining the person information corresponding to the thangka image to be identified according to the multiple groups of identification results.

[0060] When the embodiment of the present application is implemented, in order to reduce the image features in the thangka image, it is necessary to first extract the foreground image information containing only the character image, which can be extracted based on the existing mask technology, and the embodiment of the present application does not limit it. In order to accurately identify the character, the embodiment of the present application adopts a scheme of first performing human body key point recognition, wherein the limb key point information to be identified needs to include arm key point information, leg key point information, torso key point information and head key point information. For example, OpenPose is used to mark and identify the joints, and the identified key points are formed into multiple vectors. It should be understood that after the key point information is identified based on the prior art, the identified limb key point information needs to include the coordinates of the key point, the corresponding part of the key point and the vector formed between different key points. These contents belong to the prior art, and the embodiment of the present application does not repeat them. At the same time, in order to supplement the relevant features, it is necessary to capture the supplementary identification information. Specifically, the supplementary identification information generally corresponds to different limb parts, such as the gesture corresponding to the wrist, the sitting posture corresponding to the leg, and the face corresponding to the head.

[0061] In the embodiments of the present application, the inventors found that the characters in the thangka images may appear in different forms in different images, such as three heads and eight arms in one thangka image, and one head and two arms in another thangka image. Therefore, in order to improve the accuracy of recognition, the embodiments of the present application enhance the acquired arm key point information and head key point information, wherein the enhancement method is that for the arm key point information, it is necessary to refer to the relative position of the arm key point information and the torso key point information for increase or decrease calculation, while the head key point information can be enhanced only based on the position of the head key point information itself. After the enhanced information is combined, multiple groups of combined information can be formed, and after the combined information and the key point information of the legs and torso are combined, information for identification can be formed.

[0062] In the embodiment of the present application, it is necessary to pre-train the recognition model, which can be implemented based on a graph convolutional network. After the recognition model recognizes the information to be recognized, multiple results can be output, and then the characters in the thangka image can be more accurately recognized based on the distribution of these multiple results. The embodiment of the present application provides a technical solution for improving the existing human posture estimation algorithm to achieve character recognition in response to the complex human body composition problem of the characters in the thangka image through the above technical solution. It can cope with the recognition of characters in different postures in thangka, helping us to more accurately identify and understand the identity of the characters in thangka art.

[0063] In a possible implementation, capturing image features corresponding to different limbs to form supplementary identification information includes:

[0064] Capturing image features of the parts corresponding to the key point information of the arms to form gesture features, capturing image features of the parts corresponding to the key point information of the legs to form sitting posture features, and capturing image features of the parts corresponding to the key point information of the head to form facial features;

[0065] The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

[0066] When implementing the embodiment of the present application, the image features of different parts need to be captured in the supplementary identification information, generally including gesture features, sitting posture features and facial features. It should be understood that the supplementary identification information should be fixed and cannot change with the increase of arm key point information and facial key point information.

[0067] In a possible implementation, adding or reducing the arm key point information to form a plurality of enhanced arm information according to the positional relationship between the arm key point information and the torso key point information includes:

[0068] Acquire the shoulder key point information, the elbow key point information and the wrist key point information in the arm key point information, and calculate the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculate the vector direction between the wrist key point information and the elbow key point information as the lower arm direction;

[0069] When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion;

[0070] When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation;

[0071] New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

[0072] When the embodiment of the present application is implemented, for the enhancement of the arm key point information, different upper arm directions need to correspond to different lower arm directions, and the corresponding relationship can be achieved through a pre-set direction query table. In specific implementation, reducing the arm key point information needs to be reduced on both sides of the torso key point information at the same time, and when increasing the arm key point information, it is necessary to generate a new upper arm direction first. The generation process is to rotate and generate with the shoulder key point information as the axis. After the upper arm direction is generated, the lower arm direction corresponding to the upper arm direction can be queried through the direction query table to generate a new lower arm direction, thereby generating new arm key point information. It should be understood that the newly added arm key point information also needs to be generated in pairs with the torso key point information as the axis.

[0073] In a possible implementation manner, adding or reducing the head key point information according to the head key point information to form a plurality of enhanced head information includes:

[0074] When there is only one group of head key point information, taking the group as an axis, generating new head key point information of corresponding angles on both sides of the group;

[0075] When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

[0076] When the embodiment of the present application is implemented, the deletion of head key point information only exists in the cases of a single head and three heads. When adding head key point information, the newly added head key point information needs to appear in pairs on both sides of the original head key point information with a fixed angle. When reducing head key point information, head key point information on both sides is required.

[0077] In a possible implementation, the generation of the recognition model includes:

[0078] Acquire multiple thangka images and extract character images as a sample image group;

[0079] Extracting key points of a human body for the sample image group, and forming a first sample pair with the extracted key points and the names of corresponding people;

[0080] Extracting image feature information corresponding to different human key points in the sample image group as supplementary sample information, and adding the supplementary sample information to the first sample pair to form a second sample pair; the supplementary sample information includes gesture features, sitting posture features and facial features;

[0081] The recognition model is formed by training a model using the key points and supplementary sample information in the second sample pair as input information and the names of the corresponding characters in the second sample pair as output information.

[0082] When the embodiments of the present application are implemented, in the process of generating the recognition model, the powerful ability of GCN can be used to capture the complex spatial relationship between key points. In the process of training the GCN model, the network learns how to identify the identity of the characters depicted in the picture through the characteristics of the nodes and the connection patterns between them. This includes learning the unique posture patterns of different characters, such as details such as gestures and sitting postures, which are often the key clues to distinguish different characters. The training of the GCN model belongs to the mature prior art, and the embodiments of the present application do not make much limitation.

[0083] In a possible implementation, obtaining the character information corresponding to the thangka image to be identified according to the multiple sets of identification results includes:

[0084] The character information with the highest proportion is selected from the multiple groups of recognition results as the character information corresponding to the thangka image to be recognized.

[0085] When the embodiment of the present application is implemented, each thangka image to be identified will generate multiple groups of recognition results. Counting the character information with the highest proportion in these recognition results as the corresponding character information can effectively improve the recognition accuracy.

[0086] Based on the same inventive concept, the embodiment of the present application also provides a thangka image character recognition system based on human posture estimation, including:

[0087] A preprocessing unit is configured to preprocess the thangka image to be identified to obtain foreground image information containing only the image of the person;

[0088] A key point unit is configured to perform human key point recognition on the foreground image information to form limb key point information, and to capture image features corresponding to different limbs to form supplementary recognition information; the limb key point information includes arm key point information, leg key point information, torso key point information and head key point information;

[0089] an enhancement unit, configured to increase or decrease the arm key point information according to the positional relationship between the arm key point information and the torso key point information to form a plurality of enhanced arm information, and to increase or decrease the head key point information according to the head key point information to form a plurality of enhanced head information;

[0090] a combining unit configured to randomly combine the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and to combine the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified;

[0091] The recognition unit is configured to input multiple groups of information to be recognized corresponding to the thangka image to be recognized into a preset recognition model for recognition to obtain multiple groups of recognition results, and obtain the character information corresponding to the thangka image to be recognized based on the multiple groups of recognition results.

[0092] In a possible implementation, the key point unit is further configured as:

[0093] Capturing image features of the parts corresponding to the key point information of the arms to form gesture features, capturing image features of the parts corresponding to the key point information of the legs to form sitting posture features, and capturing image features of the parts corresponding to the key point information of the head to form facial features;

[0094] The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

[0095] In a possible implementation manner, the enhancement unit is further configured to:

[0096] Acquire the shoulder key point information, the elbow key point information and the wrist key point information in the arm key point information, and calculate the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculate the vector direction between the wrist key point information and the elbow key point information as the lower arm direction;

[0097] When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion;

[0098] When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation;

[0099] New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

[0100] In a possible implementation manner, the enhancement unit is further configured to:

[0101] When there is only one group of head key point information, taking the group as an axis, generating new head key point information of corresponding angles on both sides of the group;

[0102] When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

[0103] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0105] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0106] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0108] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for character recognition in thangka images based on human posture estimation, characterized in that: include: Preprocess the thangka image to be identified to obtain foreground image information containing only the person image; Performing human body key point recognition on the foreground image information to form limb key point information, and capturing image features corresponding to different limbs to form supplementary recognition information; The limb key point information includes arm key point information, leg key point information, torso key point information and head key point information; According to the positional relationship between the arm key point information and the torso key point information, the arm key point information is increased or decreased to form a plurality of enhanced arm information, and according to the head key point information, the head key point information is increased or decreased to form a plurality of enhanced head information; Randomly combining the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and combining the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified; Inputting multiple sets of information to be identified corresponding to the thangka image to be identified into a preset recognition model for identification to obtain multiple sets of identification results, and obtaining the person information corresponding to the thangka image to be identified according to the multiple sets of identification results; Capturing the image features corresponding to different limbs to form supplementary identification information includes: capturing the image features corresponding to the arm key point information to form gesture features, capturing the image features corresponding to the leg key point information to form sitting posture features, and capturing the image features corresponding to the head key point information to form facial features; The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

2. The method for character recognition in thangka images based on human posture estimation according to claim 1, characterized in that: Adding or reducing the arm key point information according to the positional relationship between the arm key point information and the torso key point information to form a plurality of enhanced arm information includes: acquiring shoulder key point information, elbow key point information and wrist key point information in the arm key point information, and calculating the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculating the vector direction between the wrist key point information and the elbow key point information as the lower arm direction; When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion; When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation; New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

3. The method for character recognition in thangka images based on human posture estimation according to claim 1, characterized in that: According to the head key point information, the head key point information is increased or decreased to form a plurality of enhanced head information, including: when there is only one group of head key point information, taking the group as an axis, generating new head key point information of corresponding angles on both sides of the group; When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

4. The method for character recognition in thangka images based on human posture estimation according to claim 1, characterized in that: The generation of the recognition model includes: acquiring a plurality of thangka images and extracting character images as a sample image group; Extracting key points of a human body for the sample image group, and forming a first sample pair with the extracted key points and the names of corresponding people; Extracting image feature information corresponding to different human key points in the sample image group as supplementary sample information, and adding the supplementary sample information to the first sample pair to form a second sample pair; the supplementary sample information includes gesture features, sitting posture features and facial features; The recognition model is formed by training a model using the key points and supplementary sample information in the second sample pair as input information and the names of the corresponding characters in the second sample pair as output information.

5. The method for thangka image character recognition based on human posture estimation according to claim 1, characterized in that: Acquiring character information corresponding to the thangka image to be identified according to the multiple groups of identification results includes: selecting character information with the highest proportion from the multiple groups of identification results as the character information corresponding to the thangka image to be identified.

6. The thangka image character recognition system based on human posture estimation is characterized by: include: A preprocessing unit is configured to preprocess the thangka image to be identified to obtain foreground image information containing only the image of the person; A key point unit is configured to perform human key point recognition on the foreground image information to form limb key point information, and capture image features corresponding to different limbs to form supplementary recognition information; The limb key point information includes arm key point information, leg key point information, torso key point information and head key point information; an enhancement unit, configured to increase or decrease the arm key point information according to the positional relationship between the arm key point information and the torso key point information to form a plurality of enhanced arm information, and to increase or decrease the head key point information according to the head key point information to form a plurality of enhanced head information; a combining unit configured to randomly combine the plurality of enhanced head information and the plurality of enhanced arm information to form a plurality of sets of head-hand combination information, and to combine the head-hand combination information, the leg key point information, the torso key point information and the corresponding supplementary identification information to form a plurality of sets of information to be identified; The recognition unit is configured to input multiple groups of information to be recognized corresponding to the thangka image to be recognized into a preset recognition model for recognition to obtain multiple groups of recognition results, and obtain the character information corresponding to the thangka image to be recognized according to the multiple groups of recognition results; The key point unit is further configured to: capture image features of the parts corresponding to the key point information of the arm to form gesture features, capture image features of the parts corresponding to the key point information of the leg to form sitting features, and capture image features of the parts corresponding to the key point information of the head to form facial features; The gesture features, sitting posture features and facial features are used as the supplementary identification information corresponding to different limbs.

7. The thangka image character recognition system based on human posture estimation according to claim 6 is characterized in that: The enhancement unit is further configured to: obtain shoulder key point information, elbow key point information and wrist key point information in the arm key point information, and calculate the vector direction between the shoulder key point information and the elbow key point information as the upper arm direction, and calculate the vector direction between the wrist key point information and the elbow key point information as the lower arm direction; When reducing the arm key point information, at least one pair of the arm key point information with the trunk key point information as an axis is deleted, and at least one pair of the arm key point information is retained after the deletion; When adding the arm key point information, the upper arm direction is rotated with the shoulder key point information as the axis, and a new lower arm direction is determined according to the angle relationship between the upper arm direction and the torso key point information after the rotation; New arm key point information is generated according to the new lower arm direction and the rotated upper arm direction, and the newly generated arm key point information is generated in pairs with the torso key point information as an axis.

8. The thangka image character recognition system based on human posture estimation according to claim 6 is characterized in that: The enhancement unit is further configured to: when there is only one group of the head key point information, generate new head key point information of corresponding angles on both sides of the group with the group as the axis; When there are only three groups of head key point information, the head key point information except for the group in the middle position is deleted.

Images