Picture resource display method and system applied to virtual space and storage medium

Abstract

The invention discloses a picture resource display method and system applied to a virtual space and a storage medium, and belongs to the technical field of virtualization and reality, and the method comprises the steps: obtaining picture resources in the current virtual space; performing hierarchical processing on the picture resources; the middle layer information is calculated as a first occupied area S1, the middle layer information is calculated as a second occupied area S2, and the bottom layer information is calculated as a third occupied area S3; detecting whether the S2 and the S3 are overlapped or not, and if not, calculating S1 + S2 as a transformation region; if yes, S-S3 is calculated as a transformation area, and S is the total area of the picture resources; and receiving a message command, converting the intermediate layer information, filling a conversion area with a conversion result, and displaying the conversion result. According to the method and the device, the text in the original picture resource is extracted for transformation by utilizing hierarchical processing on the picture resource, and the text is re-displayed on the picture resource.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual and real technologies, and particularly to a method, a system and a storage medium for displaying picture resources applied to a virtual space. Background Art

[0002] In virtual reality (VR) and augmented reality (AR) applications, users often use the resource display function in the virtual space. With the development of globalization, the communication between people becomes more and more frequent, but the language barrier has become a common problem. For example, when displaying picture resources, the patterns can be intuitively understood by users, but the text is difficult to understand because of different languages. Therefore, it is a pain point to be solved at present to facilitate users' reading and understanding and improve the convenience of users' use when displaying picture resources.

[0003] The currently common solution is to translate the materials involving text in the scene through built-in plugins or external translation software. However, this method relies on manual operation, which is time-consuming and has a poor viewing effect. When the scene is large and there is a lot of internal information, it is very inefficient to manually translate the information every time. Translation is only carried out when needed, and waiting is required during the translation process, which increases the time cost. And the manual operation realizes the translation of the text information in the image by manually selecting the area to be translated. For example, the photo translation of some translation software. First, the picture in the scene is photographed or imported, and the text extracted from the picture is used for translation feedback. The translated content is separated from the original information, which is not only cumbersome to operate but also prone to errors. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems of low efficiency and poor display effect of text transformation display in the traditional picture resources, and to provide a method, a system and a storage medium for displaying picture resources applied to a virtual space, which have the advantages of online high-efficiency automatic text display transformation, better layout ornamental property, and no influence on the background display in the original picture.

[0005] In the first aspect, the present invention realizes the above purpose through the following technical solutions. A method for displaying picture resources applied to a virtual space, the method includes the following steps: Obtain the picture resources in the current virtual space; Perform a layering process on the picture resources. The layering process includes obtaining the text elements in the picture resources and saving them as intermediate layer information, obtaining the elements that have pixel stacking with the text elements and saving them as surface layer information, and obtaining the remaining other elements in the picture resources and saving them as bottom layer information; Calculate the area of the intermediate layer information on the picture resource, set this area as the first occupied area S1, calculate the area of the intermediate layer information on the picture resource, set this area as the second occupied area S2, calculate the area of the bottom layer information on the picture resource, set this area as the third occupied area S3; Detect whether there is an overlap between S2 and S3. If not, calculate S1 + S2 as the transformation area; if so, calculate S - S3 as the transformation area, where S is the total area of the picture resource; Receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it.

[0006] Preferably, the method of obtaining the element that has pixel stacking with the text element and saving it as the surface layer information is as follows: Use an AI text recognition assistant to obtain the pixel point cluster of the text element; Perform binarization processing on the pixel point cluster, and extract the pixel points that are different from the colors of other pixels as stacked pixel points; Calculate the coordinates of the stacked pixel points, set a first detection box, and use the first detection box to identify the total elements at the corresponding coordinates in the picture resource; Exclude the text element from the total elements, and the remaining elements are used as the surface layer information.

[0007] Preferably, the method further includes extracting the pixel color and pixel spacing of the surface layer information as comparison features, setting a second detection box, identifying the elements with the same comparison features through the second detection box, and also saving them as the surface layer information, and setting the size of the second detection box to be larger than the size of the first detection box.

[0008] Preferably, the method of calculating the area of the intermediate layer information on the picture resource is as follows: Obtain the pixel point coordinates of the text element; Set a border for each pixel point, the size of the border is larger than the pixel point, and the center point of the border coincides with the center point of the pixel point; Calculate the fixed-point coordinates of all the borders, and exclude the duplicate coordinates; Obtain the point coordinates of the complete shape composed of all the borders, and calculate the area of the complete shape.

[0009] Preferably, the receiving of the message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and adding or deleting text.

[0010] Preferably, the filling of the transformation result into the transformation area and displaying it further includes that if the size of the transformation result is larger than the transformation area, a sliding component is generated at the position of the transformation area, and the sliding component is used to display the complete text content by sliding.

[0011] In a second aspect, the present invention achieves the above object through the following technical solutions. A picture resource display system applied to a virtual space, the system comprising: A resource acquisition unit, configured to acquire picture resources in the current virtual space; A picture layering unit, configured to perform layering processing on the picture resources. The layering processing includes acquiring text elements in the picture resources and saving them as intermediate layer information, acquiring elements having pixel stacking with the text elements and saving them as surface layer information, and acquiring the remaining other elements in the picture resources and saving them as bottom layer information; A region calculation unit, configured to calculate the region of the intermediate layer information on the picture resources, set this region as the first occupied region S1, calculate the region of the intermediate layer information on the picture resources, set this region as the second occupied region S2, calculate the region of the bottom layer information on the picture resources, and set this region as the third occupied region S3; A display calculation unit, configured to detect whether there is an overlap between S2 and S3. If not, calculate S1 + S2 as the transformation region; if so, calculate S - S3 as the transformation region, where S is the total region of the picture resources; A display unit, configured to receive a message command to transform the intermediate layer information, fill the transformation result into the transformation region and display it.

[0012] Preferably, the method for acquiring elements having pixel stacking with the text elements and saving them as surface layer information in the picture layering unit is as follows: Use an AI text recognition assistant to obtain the pixel point clusters of the text elements; Perform binarization processing on the pixel point clusters, and extract pixel points different from other pixel colors as stacked pixel points; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the picture resources; Exclude the text elements from the total elements, and the remaining elements are used as surface layer information.

[0013] Preferably, the picture layering unit further includes extracting the pixel color and pixel spacing of the surface layer information as comparison features, setting a second detection frame, and identifying elements having the same comparison features through the second detection frame, which are also saved as surface layer information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

[0014] In a third aspect, the present invention achieves the above object through the following technical solutions. A storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the picture resource display method applied to a virtual space as described in the first aspect.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention utilizes the hierarchical processing of picture resources, enabling the text in the original picture resources to be extracted and transformed online and redisplayed on the picture resources to meet different display requirements, such as text translation, font size adjustment, etc. Moreover, the online processing method is more efficient. When the present invention performs hierarchical processing on picture resources, the picture resources are divided into three layers, with the bottom layer as the background and the middle layer and the surface layer as transformable areas, so that the text finally displayed after transformation can correspond to the original position, without occupying the background part of the picture resources. The overall layout is more beautiful and tidy, with better viewing performance, and the text transformation accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a method for displaying picture resources applied to a virtual space according to the present invention.

[0017] Figure 2 It is a schematic diagram of a system for displaying picture resources applied to a virtual space according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] Embodiment 1 As Figure 1 shown, a method for displaying picture resources applied to a virtual space includes the following steps: Step 1: Obtain the picture resources in the current virtual space. In this step, the picture resources being displayed or about to be displayed are obtained, and they are obtained from the virtual space, rather than before the picture resources are imported into the virtual space. In this way, the picture resources displayed in the current virtual space can be obtained online, providing a data basis for subsequent processing of some picture resources, and ensuring that the original files of the picture resources will not be affected during the processing. The principle of obtaining picture resources in the virtual space is to search for the corresponding picture resources in the resource package according to the picture encoding. The obtaining method belongs to a kind of prior art, so it will not be elaborated here.

[0020] Step 2: Perform hierarchical processing on the picture resource. The hierarchical processing includes obtaining the text elements in the picture resource and saving them as intermediate layer information, obtaining the elements with pixel stacking with the text elements and saving them as surface layer information, and obtaining the remaining other elements in the picture resource and saving them as bottom layer information; this hierarchical processing is vertical hierarchical, dividing the elements originally in the same plane layer into a three-dimensional structure composed of the surface layer to the bottom layer, and using this hierarchical result to classify each element on the picture resource. For example, all elements in the processed bottom layer information are used as the background in the picture resource, and this background can be parameters such as logos, background patterns, and colors. Since the hierarchical processing realizes the judgment of each level by text elements, the above bottom layer information does not contain text, and the intermediate layer information is pure text elements. Subsequent transformations are also performed on the text elements. Because information such as patterns can be understood by different users, but text information cannot be popularly understood by users in different language systems. This solution mainly performs subsequent transformation displays on text elements. The surface layer information is the element covering the text elements. Here, the upper and lower are judged by whether the pixels of the text elements are stacked. Because pixel stacking indicates that the displayed content in this part does not need to be completely independent, and even if there is stacking, it does not affect the viewing of the text elements. For example, modification marks, annotations, etc. on the text elements, because there is pixel stacking with the text elements, so even if the text elements occupy this part of the area of the surface layer information later, it will not affect the display of the surface layer information.

[0021] Step 3: Calculate the area of the intermediate layer information on the picture resource, set this area as the first occupied area S1, calculate the area of the intermediate layer information on the picture resource, set this area as the second occupied area S2, calculate the area of the bottom layer information on the picture resource, and set this area as the third occupied area S3; in order to perform transformation display on the intermediate layer information, it is necessary to calculate the area available for the intermediate layer information to be displayed. In the original picture resource, the bottom layer information, intermediate layer information, and surface layer information each have their own occupied areas in the picture resource. After the intermediate layer information is transformed, a certain space is required for display. In order not to affect the display effects of other layers, this step calculates the occupied area of each layer. There are many ways to calculate the occupied area of elements. Usually, border processing is performed on pixel points, and other pixel points in a small range around the pixel points are included as all pixel points of each layer of information. Using the distribution of pixel points, the size of the area occupied by each layer in the picture resource is calculated.

[0022] Step 4: Detect whether there is an overlap between S2 and S3. If not, calculate S1 + S2 as the transformation area; if so, calculate S - S3 as the transformation area, where S is the total area of the picture resource. The transformation area here is as mentioned in Step 3. After the intermediate layer information is transformed, a part of the space in the picture resource needs to be selected as the display area. Since there may be a situation where the surface layer information overlaps with the bottom layer information, this is because in the layering process in Step 2, the layering is not based on the areas occupied by elements at different levels, but by classifying elements into each layer. Therefore, there may be a situation where some elements overlap with each other. According to the layering method in Step 2, only the elements between the intermediate layer information and the bottom layer information do not overlap, that is, there is no overlap between S1 and S3, and it is impossible to rule out whether there is an overlap between the elements of the bottom layer information and the surface layer information, that is, it is impossible to rule out the overlap between S2 and S3. Therefore, in this step, it is necessary to first determine whether there is an overlap between S2 and S3. If there is no overlap, then use S1 + S2 as the transformation area, which is equivalent to expanding the area where the intermediate layer information in the original picture resource can be displayed. Since the elements in the surface layer information originally overlap with the text elements in the intermediate layer information, even if the original display area is expanded, it will not affect the display effect. Here, expanding the original display area is to facilitate subsequent transformation operations on the intermediate layer information, because the original area space is too small, which will affect the complete display of the transformed text. For example, during the Chinese-to-English translation process, the length of the Chinese character string is shorter than that of the English. If it is displayed in the original area, the font must be reduced or a sliding component must be added, and the display effect is relatively poor.

[0023] Step 5: Receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it. Without transforming the bottom layer information and the surface layer information, transform the intermediate layer information and fill it in the transformation area.

[0024] In Step 2, the method of obtaining the elements with pixel stacking with the text elements and saving them as the surface layer information is as follows: Use an AI text recognition assistant to obtain the pixel point cluster of the text elements. Using an AI text recognition assistant can efficiently recognize the text elements in the picture resource. When dividing the intermediate layer, this method can also be used to improve efficiency. Recognizing the text elements is to record the pixel points of the text elements and provide a basis for subsequent calculation of the surface layer information.

[0025] Perform binarization processing on the pixel point cluster, and extract the pixel points that are different from the colors of other pixels as the stacked pixel points. Binarization processing belongs to the prior art and is a common technical means in image processing to distinguish two images with different colors. Since the pixel points of the text elements are stacked, the pixel points in the stacked part will become darker in color. Therefore, binarization processing can be used to distinguish the ordinary text element pixel points from the stacked pixel points.

[0026] Calculate the coordinates of the stacked pixel points, set the first detection box, use the first detection box to identify the total elements in the corresponding coordinates in the picture resource, and use the first detection box to include the elements around the stacked pixel points. The stacked pixel points include not only text elements but also other elements. Since this method needs to extract other elements as surface layer information, the first detection box is used to frame out other elements.

[0027] Eliminate the text elements from the total elements, and the remaining elements are used as surface layer information.

[0028] The stacking degrees of the elements of the surface layer information and the text elements are different, such as partial stacking and full stacking. The first detection box can also frame out the complete elements. However, the elements classified as surface layer information are not only the stacked parts, but there may also be elements belonging to the surface layer information around them. For example, if the surface layer information is a multi-character logo, the stacked part may be only a single character or pattern, and the subsequent other characters or patterns also belong to a type of logo. Therefore, in order not to miss the elements of the same type, the method further includes extracting the pixel color and pixel spacing of the surface layer information as comparison features, setting the second detection box, identifying the elements with the same comparison features through the second detection box, and also saving them as surface layer information. And the size of the second detection box is set to be larger than the size of the first detection box. Use the comparison features to judge whether the elements around the elements stacked with the text elements belong to the same type of elements, expand the types of elements of the surface layer information, and avoid being classified into the bottom layer information, which is likely to affect the size of the transformation area.

[0029] The method for calculating the area of the intermediate layer information on the picture resource is as follows: Obtain the pixel point coordinates of the text elements; Set a border for each pixel point. The size of the border is larger than the pixel point, and the center point of the border coincides with the center point of the pixel point; Calculate the fixed-point coordinates of all the borders and eliminate the duplicate coordinates; Obtain the point coordinates of the complete shape composed of all the borders and calculate the area of the complete shape. Since the intermediate layer information, the bottom layer information, and the surface layer information are all composed of different types of elements, the above method for calculating the area of the intermediate layer information on the picture resource can also be applied to the area calculation of the other two layers of information, only by replacing the text elements in the first step with the elements of the corresponding layer.

[0030] The receiving of the message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and addition or deletion of text. The filling of the transformation result into the transformation area and the display also includes that if the size of the transformation result is larger than the transformation area, a sliding component is generated at the position of the transformation area, and the sliding component is used to display the complete text content by sliding.

[0031] Example 2 As Figure 2 shown, a picture resource display system applied to a virtual space, the system includes: A resource acquisition unit for acquiring picture resources in the current virtual space; A picture layering unit for performing layering processing on the picture resources, where the layering processing includes obtaining text elements in the picture resources and saving them as intermediate layer information, obtaining elements with pixel stacking with the text elements and saving them as surface layer information, and obtaining the remaining other elements in the picture resources and saving them as bottom layer information; An area calculation unit for calculating the area of the intermediate layer information on the picture resources, setting this area as the first occupied area S1, calculating the area of the intermediate layer information on the picture resources, setting this area as the second occupied area S2, calculating the area of the bottom layer information on the picture resources, and setting this area as the third occupied area S3; A display calculation unit for detecting whether there is an overlap between S2 and S3. If not, calculate S1 + S2 as the transformation area; if so, calculate S - S3 as the transformation area, where S is the total area of the picture resources; A display unit for receiving a message command to transform the intermediate layer information, filling the transformation result into the transformation area and displaying it.

[0032] The method for obtaining elements with pixel stacking with the text elements and saving them as surface layer information in the picture layering unit is: Use an AI text recognition assistant to obtain the pixel point cluster of the text elements; Perform binarization processing on the pixel point cluster, and extract pixel points that are different from other pixel colors as stacked pixel points; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the picture resources; Exclude the text elements from the total elements, and the remaining elements are used as surface layer information.

[0033] The picture layering unit further includes extracting the pixel color and pixel spacing of the surface layer information as comparison features, setting a second detection frame, and identifying elements with the same comparison features through the second detection frame, which are also saved as surface layer information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

[0034] The method for calculating the area of the intermediate layer information on the picture resources is: Obtain the pixel point coordinates of the text elements; Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border coincides with the center point of the pixel. Calculate the fixed-point coordinates of all the borders and eliminate the duplicate coordinates. Obtain the point coordinates of the complete shape formed by all the borders and calculate the area of the complete shape.

[0035] The transformation of the intermediate layer information by the received message command includes text translation, font size adjustment, font transformation, and text addition and subtraction.

[0036] The filling of the transformation result into the transformation area and display further includes that if the size of the transformation result is larger than the transformation area, a sliding component is generated at the position of the transformation area. The sliding component is used to display the complete text content by sliding.

[0037] Embodiment 2 is essentially the same as Embodiment 1, so the specific working principles of each unit will not be elaborated here.

[0038] Embodiment 3 This embodiment provides a storage medium, which includes a storage program area and a storage data area. The storage program area can store an operating system, programs required to run the instant messaging function, etc.; the storage data area can store various instant messaging information and operation instruction sets, etc. The computer program is stored in the storage program area. When the computer program is executed by a processor, it implements the method for displaying picture resources applied to the virtual space as described in Embodiment 1. The processor may include one or more central processing units, or be a digital processing unit, etc.

[0039] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention.

[0040] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for displaying picture resources in a virtual space, characterized in that: The method comprises the following steps: Get the image resources in the current virtual space; The image resource is subjected to layered processing, wherein the layered processing includes obtaining text elements in the image resource and saving them as middle layer information, obtaining elements with pixel stacking with the text elements and saving them as surface layer information, and obtaining other remaining elements in the image resource and saving them as bottom layer information; Calculate the area of ​​the intermediate layer information on the picture resource and set the area as the first occupied area S1, calculate the area of ​​the intermediate layer information on the picture resource and set the area as the second occupied area S2, calculate the area of ​​the bottom layer information on the picture resource and set the area as the third occupied area S3; Detect whether S2 and S3 overlap, if not, calculate S1+S2 as the transformation area; if so, calculate S-S3 as the transformation area, where S is the total area of ​​the picture resource; The intermediate layer information is transformed upon receiving a message command, and the transformation result is filled into the transformation area and displayed.

2. The method for displaying picture resources in a virtual space according to claim 1, characterized in that: The method for obtaining the element having pixel stacking with the text element and saving it as surface information is: Use AI text recognition assistant to obtain pixel clusters of text elements; Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements in the corresponding coordinates in the image resource; Text elements are removed from the total elements, and the remaining elements are used as surface information.

3. The method for displaying picture resources in a virtual space according to claim 2, characterized in that: The method also includes extracting pixel color and pixel spacing of surface information as comparison features, setting a second detection frame, identifying elements identical to the comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than that of the first detection frame.

4. The method for displaying picture resources in a virtual space according to claim 1, characterized in that: The method for calculating the area of ​​the intermediate layer information on the picture resource is: Get the pixel coordinates of the text element; Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border is consistent with the center point of the pixel. Calculate the fixed-point coordinates of all borders and eliminate duplicate coordinates; Get the point coordinates of the complete shape composed of all borders and calculate the area of ​​the complete shape.

5. The method for displaying picture resources in a virtual space according to claim 1, characterized in that: The receiving message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and adding or removing text.

6. The method for displaying picture resources in a virtual space according to claim 5, characterized in that: Filling the transformation result into the transformation area and displaying the transformation result also includes generating a sliding component at the transformation area position if the size of the transformation result is larger than the transformation area, and the sliding component is used to display the complete text content by sliding.

7. A picture resource display system applied to a virtual space, characterized in that: The system includes: A resource acquisition unit, used to obtain image resources in the current virtual space; A picture layering unit, used for performing layering processing on the picture resource, wherein the layering processing includes obtaining text elements in the picture resource and saving them as middle layer information, obtaining elements with pixel stacking with the text elements and saving them as surface layer information, and obtaining other remaining elements in the picture resource and saving them as bottom layer information; an area calculation unit, configured to calculate an area of ​​the intermediate layer information on the picture resource, and set the area as a first occupied area S1, calculate an area of ​​the intermediate layer information on the picture resource, and set the area as a second occupied area S2, and calculate an area of ​​the bottom layer information on the picture resource, and set the area as a third occupied area S3; A display calculation unit is used to detect whether the S2 and the S3 overlap, and if not, calculate S1+S2 as the transformation area; if so, calculate S-S3 as the transformation area, where S is the total area of ​​the picture resource; The display unit is used to receive the message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it.

8. The image resource display system for virtual space according to claim 7, characterized in that: The method for obtaining the element having pixel stacking with the text element in the picture layer unit and saving it as surface information is as follows: Use AI text recognition assistant to obtain pixel clusters of text elements; Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements in the corresponding coordinates in the image resource; Text elements are removed from the total elements, and the remaining elements are used as surface information.

9. The image resource display system for virtual space according to claim 8, characterized in that: The image layering unit also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

10. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for displaying picture resources applied to a virtual space as described in claims 1-6 is implemented.

Images