Picture resource display method and system applied to virtual space, and storage medium

Abstract

The application discloses a picture resource display method and system applied to a virtual space, a storage medium and belongs to the technical field of virtual and real technologies, and the method comprises the steps of: acquiring picture resources in a current virtual space; performing hierarchical processing on the picture resources; calculating intermediate layer information as a first occupied area S1, calculating surface layer information as a second occupied area S2, and calculating bottom layer information as a third occupied area S3; detecting whether the S2 and the S3 overlap, if not, calculating S1+S2 as a transformation area; if yes, calculating S-S3 as a transformation area, wherein S is the total area of the picture resources; receiving a message command to transform the intermediate layer information, filling the transformation result to the transformation area and displaying. The application utilizes hierarchical processing on the picture resources, so that the text in the original picture resources is extracted for transformation and is newly displayed on the picture resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of virtual and real technology, in particular to a picture resource display method and system applied to a virtual space and a storage medium. BACKGROUND

[0002] In virtual reality (VR) and augmented reality (AR) applications, users often use resource display functions in virtual spaces. With the development of globalization, communication between people is becoming more and more frequent, but language barriers have become a common problem. For example, in picture resource display, patterns can be intuitively understood by users, but it is more difficult to understand text due to language differences. Therefore, it is currently a pain point to facilitate user reading and understanding in picture resource display and to improve user convenience.

[0003] A common solution at present is to translate materials involving text in a scene through built-in plug-ins or external translation software. However, this method relies on manual operation, is time-consuming and has poor viewing effect. When the scene is large and has many internal materials, it is very inefficient to manually translate the materials each time. The translation needs to be performed when needed, and the translation process needs to be waited for, increasing the time cost. And manual operation realizes the translation of text materials in the image by manually selecting the area to be translated. For example, some translation software has a photograph translation function. The scene picture is first photographed or imported, and the text extracted from the picture is translated and fed back. The translated content is separated from the original material, which is not only cumbersome to operate, but also prone to errors. SUMMARY

[0004] The purpose of the present application is to solve the problems of low efficiency and poor display effect of text transformation in traditional picture resources, and to propose a picture resource display method and system applied to a virtual space and a storage medium, which has the advantages of online efficient automatic text display, better layout and observation, and does not affect the background display of the original picture.

[0005] In a first aspect, the present application achieves the above-mentioned purpose by the following technical solution, a picture resource display method applied to a virtual space, the method comprising the following steps:

[0006] obtaining a picture resource in a current virtual space;

[0007] performing hierarchical processing on the picture resource, the hierarchical processing comprising saving text elements in the picture resource as intermediate layer information, saving elements having pixel stacking with the text elements as surface layer information, and saving remaining other elements in the picture resource as bottom layer information;

[0008] Calculate the area of ​​the intermediate layer information on the image resource and set the area as a first occupied area S1; calculate the area of ​​the surface layer information on the image resource and set the area as a second occupied area S2; calculate the area of ​​the bottom layer information on the image resource and set the area as a third occupied area S3;

[0009] Detect whether S2 and S3 overlap. If not, calculate S1+S2 as the conversion area; if so, calculate S-S3 as the conversion area, where S is the total area of ​​the image resource;

[0010] The intermediate layer information is transformed upon receiving a message command, and the transformation result is filled into the transformation area and displayed.

[0011] Preferably, the method for obtaining the element having pixel stacking with the text element and saving it as surface information is:

[0012] Use AI text recognition assistant to obtain pixel clusters of text elements;

[0013] Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels;

[0014] Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the image resource;

[0015] Remove text elements from the total elements, and use the remaining elements as surface information.

[0016] Preferably, the method also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

[0017] Preferably, the method for calculating the area of ​​the intermediate layer information on the image resource is:

[0018] Get the pixel coordinates of the text element;

[0019] Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border is consistent with the center point of the pixel.

[0020] Calculate the fixed-point coordinates of all bounding boxes and eliminate duplicate coordinates;

[0021] Get the point coordinates of the complete shape composed of all bounding boxes and calculate the area of ​​the complete shape.

[0022] Preferably, the receiving message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and adding or removing text.

[0023] Preferably, filling the transformation result into the transformation area and displaying the transformation result further includes generating a sliding component at the transformation area position if the size of the transformation result is larger than the transformation area, the sliding component being used to display the complete text content by sliding.

[0024] In a second aspect, the present invention achieves the above-mentioned purpose through the following technical solutions: a picture resource display system for a virtual space, the system comprising:

[0025] Resource acquisition unit, used to obtain image resources in the current virtual space;

[0026] An image layering unit is configured to perform layering processing on the image resource, wherein the layering processing includes obtaining text elements in the image resource and saving them as intermediate layer information, obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information, and obtaining other remaining elements in the image resource and saving them as bottom layer information;

[0027] an area calculation unit, configured to calculate an area of ​​the intermediate layer information on the image resource and set the area as a first occupied area S1, calculate an area of ​​the surface layer information on the image resource and set the area as a second occupied area S2, and calculate an area of ​​the bottom layer information on the image resource and set the area as a third occupied area S3;

[0028] A display calculation unit is configured to detect whether S2 and S3 overlap, and if not, calculate S1+S2 as the conversion area; if so, calculate S-S3 as the conversion area, where S is the total area of ​​the image resource;

[0029] The display unit is used to receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it.

[0030] Preferably, the method for obtaining the element having pixel stacking with the text element and saving it as surface information in the image layering unit is:

[0031] Use AI text recognition assistant to obtain pixel clusters of text elements;

[0032] Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels;

[0033] Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the image resource;

[0034] Remove text elements from the total elements, and use the remaining elements as surface information.

[0035] Preferably, the image layering unit also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

[0036] In a third aspect, the present invention achieves the above-mentioned purpose through the following technical solution: a storage medium storing a computer program, which, when executed by a processor, implements the image resource display method applied to a virtual space as described in the first aspect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention utilizes layered processing of image resources so that the text in the original image resource can be extracted and transformed online and redisplayed on the image resource to meet different display requirements, such as text translation, font size adjustment, etc., and the online processing method is more efficient. When the present invention processes the image resources in layers, the image resources are divided into three layers, with the bottom layer as the background, and the middle layer and the surface layer as the transformable area, so that the text that is finally transformed and displayed can correspond to the original position and will not occupy the background part of the image resource. The overall layout is more beautiful and neat, the viewing experience is better, and the text transformation accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the method for displaying image resources in a virtual space according to the present invention.

[0040] Figure 2 Schematic diagram of a picture resource display system applied to a virtual space according to the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, a method for displaying image resources in a virtual space includes the following steps:

[0044] Step 1: Obtain the image resources in the current virtual space. In this step, the image resources that are being displayed or are about to be displayed are obtained, and they are obtained from the virtual space, not before the image resources are imported into the virtual space. In this way, the image resources displayed in the current virtual space can be obtained online, and a data basis is provided for the subsequent processing of some image resources. During the processing, it is ensured that the original files of the image resources will not be affected. The principle of obtaining image resources in the virtual space is to search for the corresponding image resources in the resource package based on the image encoding. The acquisition method belongs to one of the existing technologies, so it will not be described in detail.

[0045] Step 2: perform layered processing on the image resource. The layered processing includes obtaining the text elements in the image resource and saving them as middle layer information, obtaining the elements with pixel stacking with the text elements and saving them as surface layer information, and obtaining the remaining elements in the image resource and saving them as bottom layer information. The layered processing is vertical layering, which divides the elements originally in the same plane layer into a three-dimensional structure consisting of the surface layer to the bottom layer, and uses this layered result to classify the various elements on the image resource. For example, all elements in the processed bottom layer information are used as the background of the image resource. This background can be parameters such as logo, background pattern, color, etc. Since the layered processing relies on text elements to realize the judgment of each level, the above bottom layer information does not contain text, and the middle layer information It is a pure text element, and subsequent transformations are also performed on text elements, because information such as patterns can be understood by different users, but if the text information exists in users with different language systems, it cannot be understood in a popular way. This solution also mainly performs subsequent transformation and display on text elements. Surface information is the element covering the text element. Here, the upper and lower parts are judged by whether the pixels of the text elements are stacked. Because there is pixel stacking, it means that the display content of this part does not need to be completely independent. Even if there is stacking, it does not affect the viewing of the text elements. For example, the modification traces, annotations and other content on the text elements, due to the existence of pixel stacking with the text elements, even if the text elements occupy part of the surface information area, it will not affect the display of the surface information.

[0046] Step three, calculate the area of ​​the middle layer information on the image resource, set the area as the first occupied area S1, calculate the area of ​​the surface layer information on the image resource, set the area as the second occupied area S2, calculate the area of ​​the bottom layer information on the image resource, set the area as the third occupied area S3; in order to transform and display the middle layer information, it is necessary to calculate the area available for display of the middle layer information. In the original image resource, the bottom layer information, middle layer information and surface layer information all have their own occupied areas in the image resource. After the middle layer information is transformed, a certain space is required for display. In order not to affect the display effect of other layers, this step calculates the occupied area of ​​each layer. There are many ways to calculate the occupied area of ​​elements. Usually, the pixel points are framed, and other pixel points in a small range around the pixel points are included as all pixel points of each layer of information. The distribution of pixel points is used to calculate the size of the area occupied by each layer in the image resource.

[0047] Step 4, check whether the S2 and S3 overlap, if not, calculate S1+S2 as the transformation area; if so, calculate S-S3 as the transformation area, where S is the total area of ​​the image resource; the transformation area here is as mentioned in step 3. After the intermediate layer information is transformed, a part of the space needs to be selected in the image resource as the display area. Since the surface information may be stacked with the underlying information, this is because during the layered processing in step 2, the layering is not based on the area occupied by elements of different levels, but each layer is classified by elements, so some elements will be stacked with each other. According to the layered processing method in step 2, there is no stacking between the elements of the intermediate layer information and the underlying information, that is, there is no overlap between S1 and S3, and the underlying information cannot be excluded. Whether there is overlap between the elements of the information and the surface information, that is, the overlap between S2 and S3 cannot be ruled out, so in this step it is necessary to first determine whether there is overlap between S2 and S3. If there is no overlap, S1+S2 is used as the transformation area, which is equivalent to expanding the area where the intermediate layer information in the original image resource can be displayed. Because the elements in the surface information are originally stacked with the text elements in the intermediate layer information, even if the original display area is expanded, it will not affect the display effect. The original display area is expanded here to facilitate the subsequent transformation operation of the intermediate layer information, resulting in the original area space being too small, affecting the complete display of the transformed text. For example, in the process of Chinese-English translation, the length of the Chinese character string is shorter than that of English. If it is displayed in the original area, the font must be reduced or a sliding component must be added, so the display effect is poor.

[0048] Step 5: Receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it. When the bottom layer information and the surface layer information are not transformed, transform the intermediate layer information and fill it into the transformation area.

[0049] In step 2, the method of obtaining the element having pixel stacking with the text element and saving it as surface information is as follows:

[0050] Use the AI ​​text recognition assistant to obtain the pixel clusters of text elements. Using the AI ​​text recognition assistant, you can efficiently identify text elements in image resources. This method can also be used to improve efficiency when dividing the middle layer. The purpose of identifying text elements is to record the pixel points of text elements, which also provides a basis for subsequent calculation of surface information.

[0051] The pixel clusters are binarized and pixels with colors different from other pixels are extracted as stacked pixels. Binarization belongs to the existing technology and is a common technical means for distinguishing two images of different colors in image processing. Since the pixels of text elements are stacked, the color of the pixels in the stacked part will become darker, so binarization can be used to distinguish between ordinary text element pixels and stacked pixels.

[0052] Calculate the coordinates of the stacked pixel points, set the first detection box, use the first detection box to identify the total elements in the corresponding coordinates in the image resource, and use the first detection box to include the elements around the stacked pixel points. The stacked pixel points include not only text elements, but also other elements. This method needs to extract other elements as surface information, so the first detection box is used to select other elements.

[0053] Remove text elements from the total elements, and use the remaining elements as surface information.

[0054] The elements of surface information have different degrees of stacking with text elements, such as partial stacking and full stacking. The first detection frame can also be used to select the complete element, but the elements classified as surface information are not only the stacked parts, but there may also be elements belonging to the surface information around them. For example, the surface information is a multi-character logo, then the stacked part may be just a single character or pattern, and the subsequent other characters or patterns also belong to a type of logo. In order not to miss elements of the same type, the method also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame larger than the size of the first detection frame, using the comparison features to determine whether the elements stacked with the text elements belong to the same type of elements, expanding the types of elements of the surface information, and avoiding being divided into the underlying information, which easily affects the size of the transformation area.

[0055] The method for calculating the area of ​​the intermediate layer information on the image resource is:

[0056] Get the pixel coordinates of the text element;

[0057] Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border is consistent with the center point of the pixel.

[0058] Calculate the fixed-point coordinates of all bounding boxes and eliminate duplicate coordinates;

[0059] Obtain the coordinates of all points that form the complete shape of the bounding box and calculate the area of ​​the complete shape. Since the middle layer, bottom layer, and surface layer information are all composed of different types of elements, the above method for calculating the area of ​​the middle layer information on the image resource can also be applied to the area calculation of the other two layers of information. Simply replace the text elements in the first step with elements from the corresponding layers.

[0060] The received message command transforms the intermediate layer information, including text translation, font size adjustment, font transformation, and adding or subtracting text. The filling of the transformation result into the transformation area and displaying it also includes generating a sliding component at the transformation area position if the size of the transformation result is larger than the transformation area. The sliding component is used to display the complete text content by sliding.

[0061] Example 2

[0062] like Figure 2 As shown, a picture resource display system applied to a virtual space includes:

[0063] Resource acquisition unit, used to obtain image resources in the current virtual space;

[0064] An image layering unit is configured to perform layering processing on the image resource, wherein the layering processing includes obtaining text elements in the image resource and saving them as intermediate layer information, obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information, and obtaining other remaining elements in the image resource and saving them as bottom layer information;

[0065] an area calculation unit, configured to calculate an area of ​​the intermediate layer information on the image resource and set the area as a first occupied area S1, calculate an area of ​​the surface layer information on the image resource and set the area as a second occupied area S2, and calculate an area of ​​the bottom layer information on the image resource and set the area as a third occupied area S3;

[0066] A display calculation unit is configured to detect whether S2 and S3 overlap, and if not, calculate S1+S2 as the conversion area; if so, calculate S-S3 as the conversion area, where S is the total area of ​​the image resource;

[0067] The display unit is used to receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it.

[0068] The method for obtaining the element having pixel stacking with the text element in the image layer unit and saving it as surface layer information is as follows:

[0069] Use AI text recognition assistant to obtain pixel clusters of text elements;

[0070] Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels;

[0071] Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the image resource;

[0072] Remove text elements from the total elements, and use the remaining elements as surface information.

[0073] The image layering unit also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

[0074] The method for calculating the area of ​​the intermediate layer information on the image resource is:

[0075] Get the pixel coordinates of the text element;

[0076] Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border is consistent with the center point of the pixel.

[0077] Calculate the fixed-point coordinates of all bounding boxes and eliminate duplicate coordinates;

[0078] Get the point coordinates of the complete shape composed of all bounding boxes and calculate the area of ​​the complete shape.

[0079] The receiving message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and adding or removing text.

[0080] Filling the transformation result into the transformation area and displaying the result further includes generating a sliding component at the position of the transformation area if the size of the transformation result is larger than the transformation area, and the sliding component is used to display the complete text content by sliding.

[0081] Example 2 is essentially the same as Example 1, so the specific working principles of each unit are not described in detail.

[0082] Example 3

[0083] This embodiment provides a storage medium comprising a program storage area and a data storage area. The program storage area can store an operating system and programs required for instant messaging functions, while the data storage area can store various instant messaging messages and operating instruction sets. A computer program is stored in the program storage area. When executed by a processor, the method for displaying image resources in a virtual space as described in Example 1 is implemented. The processor can include one or more central processing units, a digital processing unit, or the like.

[0084] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations that come within the meaning and range of equivalents of the claims be embraced therein.

[0085] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for displaying image resources in a virtual space, characterized in that: The method comprises the following steps: Get the image resources in the current virtual space; The image resource is subjected to layered processing, wherein the layered processing includes obtaining text elements in the image resource and saving them as intermediate layer information, obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information, and obtaining other remaining elements in the image resource and saving them as bottom layer information. The method for obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information is as follows: Use AI text recognition assistant to obtain pixel clusters of text elements; Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the image resource; Remove text elements from the total elements, and use the remaining elements as surface information; Calculate the area of ​​the intermediate layer information on the image resource and set the area as a first occupied area S1; calculate the area of ​​the surface layer information on the image resource and set the area as a second occupied area S2; calculate the area of ​​the bottom layer information on the image resource and set the area as a third occupied area S3; Detect whether S2 and S3 overlap. If not, calculate S1+S2 as the conversion area; if so, calculate S-S3 as the conversion area, where S is the total area of ​​the image resource; The intermediate layer information is transformed upon receiving a message command, and the transformation result is filled into the transformation area and displayed.

2. The method for displaying image resources in a virtual space according to claim 1, wherein: The method also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

3. The method for displaying image resources in a virtual space according to claim 1, wherein: The method for calculating the area of ​​the intermediate layer information on the image resource is: Get the pixel coordinates of the text element; Set a border for each pixel. The size of the border is larger than the pixel, and the center point of the border is consistent with the center point of the pixel. Calculate the fixed-point coordinates of all bounding boxes and eliminate duplicate coordinates; Get the point coordinates of the complete shape composed of all bounding boxes and calculate the area of ​​the complete shape.

4. The method for displaying image resources in a virtual space according to claim 1, wherein: The receiving message command to transform the intermediate layer information includes text translation, font size adjustment, font transformation, and adding or removing text.

5. The method for displaying picture resources in a virtual space according to claim 4, characterized in that: Filling the transformation result into the transformation area and displaying the result further includes generating a sliding component at the transformation area if the size of the transformation result is larger than the transformation area, the sliding component being used to display the complete text content by sliding.

6. A picture resource display system applied to a virtual space, characterized in that: The system includes: Resource acquisition unit, used to obtain image resources in the current virtual space; The image layering unit is configured to perform layering processing on the image resource, wherein the layering processing includes obtaining text elements in the image resource and saving them as intermediate layer information, obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information, and obtaining other remaining elements in the image resource and saving them as bottom layer information. The method for obtaining elements that are pixel-stacked with the text elements and saving them as surface layer information in the image layering unit is as follows: Use AI text recognition assistant to obtain pixel clusters of text elements; Binarize the pixel clusters and extract the pixels that are different from other pixel colors as stacked pixels; Calculate the coordinates of the stacked pixel points, set a first detection frame, and use the first detection frame to identify the total elements at the corresponding coordinates in the image resource; Remove text elements from the total elements, and use the remaining elements as surface information; an area calculation unit, configured to calculate an area of ​​the intermediate layer information on the image resource and set the area as a first occupied area S1, calculate an area of ​​the surface layer information on the image resource and set the area as a second occupied area S2, and calculate an area of ​​the bottom layer information on the image resource and set the area as a third occupied area S3; A display calculation unit is configured to detect whether S2 and S3 overlap, and if not, calculate S1+S2 as the conversion area; if so, calculate S-S3 as the conversion area, where S is the total area of ​​the image resource; The display unit is used to receive a message command to transform the intermediate layer information, fill the transformation result into the transformation area and display it.

7. The image resource display system for virtual space according to claim 6, characterized in that: The image layering unit also includes extracting the pixel color and pixel spacing of the surface information as comparison features, setting a second detection frame, identifying elements with the same comparison features through the second detection frame, and also saving them as surface information, and setting the size of the second detection frame to be larger than the size of the first detection frame.

8. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for displaying image resources applied to a virtual space as claimed in any one of claims 1 to 5 is implemented.

Images