An image data processing method, device, and computer-readable storage medium

Abstract

The present application discloses an image data processing method, apparatus, and computer-readable storage medium. The method includes: obtaining a first object image area where a virtual user object is located in an image; the first object image area is obtained according to similar virtual attribute objects corresponding to a virtual user object cluster; the virtual user object cluster includes virtual user objects; there is a relative position relationship between the virtual user object and the similar virtual attribute objects in the image; obtaining multimedia data for independent display on the image; performing hidden display on the multimedia data in the first object image area, and performing superimposed display on the multimedia data in the background image area and the background image area; the background image area is the image area in the image other than the first object image area. By using the present application, when displaying multimedia data on an image, the display effect of the virtual user object in the image can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and particularly to an image data processing method, apparatus, and computer-readable storage medium. Background Art

[0002] In the existing game live broadcast scenario, the comment remarks entered by users can be displayed in the game live broadcast screen, and the comment remarks can be displayed in the form of bullet screens in the game live broadcast screen. Since a game faces a large number of users, each user can enter bullet screens when watching the game live broadcast. Therefore, the number of bullet screens displayed in the game live broadcast screen will be extremely large.

[0003] In the prior art, usually all bullet screens are directly displayed flowing in the game live broadcast screen. Therefore, these numerous bullet screens are very likely to block the game characters in the game live broadcast screen, resulting in poor display effects of each game character in the game live broadcast screen. Summary of the Invention

[0004] The present application provides an image data processing method, apparatus, and computer-readable storage medium, which can improve the display effect of virtual user objects in an image when displaying multimedia data on the image.

[0005] On the one hand, the present application provides an image data processing method, including:

[0006] Obtain a first object image area where a virtual user object is located in the image; the first object image area is obtained according to similar virtual attribute objects corresponding to a virtual user object cluster; the virtual user object cluster includes virtual user objects; the virtual user object and the similar virtual attribute object have a relative position relationship in the image;

[0007] Obtain multimedia data for independent display on the image;

[0008] Hide and display the multimedia data in the first object image area, and superimpose and display the multimedia data in the background image area and the background image area; the background image area is the image area in the image except the first object image area.

[0009] On the one hand, the present application provides an image data processing apparatus, including:

[0010] An identification module, configured to obtain a first object image area where a virtual user object is located in the image; the first object image area is obtained according to similar virtual attribute objects corresponding to a virtual user object cluster; the virtual user object cluster includes virtual user objects; the virtual user object and the similar virtual attribute object have a relative position relationship in the image;

[0011] An acquisition module, configured to acquire multimedia data for independent display on an image;

[0012] A display module, configured to perform hidden display on the multimedia data in the first object image area, and perform superimposed display on the multimedia data in the background image area and the background image area; the background image area is the image area in the image except the first object image area.

[0013] Wherein, the recognition module includes:

[0014] A matching acquisition unit, configured to acquire a matching image containing a similar virtual attribute object;

[0015] A traversal unit, configured to slide and traverse the matching image in the image to obtain a matching image area corresponding to the matching image in the image; the matching image area and the matching image have image similarity;

[0016] An area determination unit, configured to determine the first object image area according to the matching image area.

[0017] Wherein, the traversal unit includes:

[0018] A ratio acquisition subunit, configured to acquire a first scaling ratio corresponding to the image and a second scaling ratio corresponding to the matching image;

[0019] A ratio determination subunit, configured to acquire a unified scaling ratio if the first scaling ratio is not equal to the second scaling ratio;

[0020] A scaling subunit, configured to scale the image and the matching image according to the unified scaling ratio to obtain a scaled image and a scaled matching image;

[0021] A first traversal subunit, configured to slide and traverse the scaled matching image on the scaled image to obtain the matching image area.

[0022] Wherein, the traversal unit includes:

[0023] A step size acquisition subunit, configured to acquire a sliding step size, and slide and traverse the matching image in the image based on the sliding step size;

[0024] An overlay acquisition subunit, configured to acquire the sliding position after the matching image is slid and traversed, and acquire the overlay area image corresponding to the matching image at the sliding position in the image;

[0025] A similarity matching subunit, configured to perform image similarity matching on the overlay area image and the matching image to obtain an image matching degree;

[0026] A first region determination subunit, configured to, when the image matching degree is greater than or equal to an image matching degree threshold, determine the image region pointed to by the covered region image in the image as a matching image region.

[0027] Among them, the traversal unit includes:

[0028] A grayscale image acquisition subunit, configured to acquire a first grayscale image corresponding to the image and a second grayscale image corresponding to the matching image;

[0029] A second traversal subunit, configured to perform a sliding traversal of the second grayscale image in the first grayscale image to obtain a matching image region corresponding to the matching image.

[0030] Among them, the virtual user object is a game character object; the similar virtual attribute object is a health bar object corresponding to the game character object; the matching image is an image including the health bar object.

[0031] Among them, the matching image region includes a first image matching region and a second image matching region;

[0032] The region determination unit includes:

[0033] A position acquisition subunit, configured to acquire a first matching position of the first matching image region in the image and a second matching position of the second matching image region in the image;

[0034] A first target determination subunit, configured to, when the position distance between the first matching position and the second matching position is less than or equal to a distance threshold, determine the first matching position or the second matching position as a target position, and determine a first object image region according to the target position;

[0035] A second target determination subunit, configured to, when the position distance between the first matching position and the second matching position is greater than the distance threshold, determine both the first matching position and the second matching position as target positions, and determine a first object image region according to the target positions.

[0036] Among them, the first target determination subunit includes:

[0037] A region acquisition subunit, configured to, when there are at least two target positions, acquire a position image region corresponding to each target position in the image; the position image region is the region where the virtual user object is located;

[0038] A region merging subunit, configured to perform region merging on the position image regions corresponding to each target position to obtain a first object image region.

[0039] Among them, the first target determination subunit includes:

[0040] A first acquisition subunit, configured to acquire the object width of a similar virtual attribute object and the object length of the similar virtual attribute object;

[0041] A second region determination subunit, configured to determine a second object image region of the similar virtual attribute object in the image according to the target position, the object width of the similar virtual attribute object, and the object length of the similar virtual attribute object;

[0042] A third region determination subunit, configured to determine a first object image region according to the second object image region and the virtual user object.

[0043] Wherein, the third region determination subunit includes:

[0044] A second acquisition subunit, configured to acquire the object height of the virtual user object and the object length of the virtual user object;

[0045] A fourth region determination subunit, configured to determine the first object image region according to the object height of the virtual user object, the object length of the virtual user object, the region position of the second object image region in the image, and the relative position of the virtual user object and the second object image region in the image.

[0046] Wherein, the display module includes:

[0047] A mask setting unit, configured to set the first object image region as a mask image region;

[0048] A hidden display unit, configured to perform hidden display on the multimedia data in the first object image region based on the mask image region.

[0049] On the one hand, the present application provides a computer device, including a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor is caused to execute the method in one aspect of the present application.

[0050] On the one hand, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by the processor, the processor is caused to execute the method in the above-mentioned one aspect.

[0051] This application obtains the first object image area where the virtual user object is located in the image; the first object image area is obtained based on the similar virtual attribute objects corresponding to the virtual user object cluster; the virtual user object cluster includes virtual user objects; the virtual user object and the similar virtual attribute objects have a relative positional relationship in the image; obtains multimedia data for independent display on the image; hides and displays the multimedia data in the first object image area, and superimposes and displays the multimedia data in the background image area and the background image area; the background image area is the image area in the image except the first object image area. It can be seen that the method proposed in this application can identify the image area (i.e., the first object image area) where the virtual user object is located in the image through the similar virtual attribute objects of the virtual user object, so as to realize that when displaying multimedia data in the image, the multimedia data will not block the virtual user object in the image. Brief Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions in this application or the prior art, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0053] Figure 1 is a schematic diagram of a system architecture provided by this application;

[0054] Figure 2a is a schematic diagram of a scenario for area recognition provided by this application;

[0055] Figure 2b is a schematic diagram of a page of a terminal provided by this application;

[0056] Figure 3 is a schematic flowchart of an image data processing method provided by this application;

[0057] Figure 4 is a schematic diagram of a scenario for obtaining a target position provided by this application;

[0058] Figure 5 is a schematic diagram of a scenario for obtaining an object image area provided by this application;

[0059] Figure 6 is a schematic diagram of a scenario for area merging provided by this application;

[0060] Figure 7 is a schematic diagram of another page of a terminal provided by this application;

[0061] Figure 8It is a schematic structural diagram of an image data processing device provided by the present application;

[0062] Figure 9 It is a schematic structural diagram of a computer device provided by the present application. Specific embodiments

[0063] Next, the technical solutions in the present application will be clearly and completely described in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0064] Please refer to Figure 1 , which is a schematic diagram of a system architecture provided by the present application. As Figure 1 shown, the schematic diagram of the system architecture includes a server 100 and multiple terminal devices. The multiple terminal devices specifically include a terminal device 200a, a terminal device 200b, and a terminal device 200c. Among them, the terminal device 200a, the terminal device 200b, and the terminal device 200c can all communicate with the server 100 through the network. The terminal device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a mobile internet device (MID), a wearable device (such as a smart watch, a smart bracelet, etc.). Here, the communication between the terminal device 200a, the terminal device 200b, and the server 100 is taken as an example for description.

[0065] The terminal device 200a includes a live broadcast client, and the anchor can perform game live broadcast through the live broadcast client. The terminal device 200b includes an audience client, and the audience can watch the anchor's game live broadcast through the audience client. During the process of the anchor performing game live broadcast through the live broadcast client, the terminal device 200a can capture each frame of the game screen in the game live broadcast, and each frame of the game screen corresponds to a game screen image (which can be called a frame of game screen image). Since the present solution needs to ensure that the barrage in the game screen does not block the game characters (i.e., game figures), it is necessary to identify the image area where the game characters are located in each game screen image, and the image area needs to be set as a mask image area. Among them, the mask image area refers to the protected image area. Subsequently, when displaying the barrage in the game screen image, the barrage can not be displayed in the mask image area. Among them, identifying the image area where the game characters are located in each game screen image can be executed by the terminal device 200a or by the server 100. In other words, obtaining the mask image area in each game screen image can be executed by the terminal device 200a or by the server 100.

[0066] If the terminal device 200a is used to identify and obtain the masked image area in each game screen image, then subsequently, the terminal device 200a can send each obtained game screen image and the identified masked image area in each game screen image (including the area size and area position of the masked image area) to the server 100. Then, the server 100 can send each obtained game screen image and the masked image area in each game screen image to the terminal device 200b. The terminal device 200b can, according to the masked image area in each obtained game screen image, display each game screen image and the obtained barrage (referring to the user's comment remarks) in the viewer client. The barrage is not displayed in the masked image area of each game screen image (which can be understood as hidden display), and the barrage is normally displayed in the image area of each game screen image except for the masked image area.

[0067] If the server 100 is used to identify and obtain the masked image area in each game screen image, then first, the terminal device 200a can send each frame of game screen image obtained through the live client to the server 100. The server 100 can identify the image where the game character is located in each frame of game screen image. Similarly, the masked image area in each frame of game screen image can be obtained. Then, the server 100 can send the obtained each frame of game screen image and the identified masked image area in each frame of game screen image to the terminal device 200b. The terminal device 200b can, according to the masked image area in each obtained game screen image, display each game screen image and the obtained barrage (referring to the user's comment remarks) in the viewer client. The barrage is not displayed in the masked image area of each game screen image (which can be understood as hidden display), and the barrage is normally displayed in the image area of each game screen image except for the masked image area.

[0068] Among them, this application mainly describes how to identify the image area where the game character is located in the game screen image. The following Figure 2a corresponding embodiments specifically describe how to identify the image area where the game character is located in the game screen image. Since the methods for identifying the image area where the game character is located in each game screen image are the same and independent of each other, therefore, Figure 2a the corresponding embodiments identify the image area where the game character is located in one game screen image. Figure 2a The execution entity in the corresponding embodiments can be the live client in the terminal device 200a or the server 100. In fact, Figure 2aThe execution entity in the corresponding embodiment may also be the viewer client in the terminal device 200b. Generally, the live client in the terminal device 200a or the server 100 is used as the execution entity for identifying the image area where the game character is located in the game screen image.

[0069] Please refer to Figure 2a , which is a schematic diagram of a scene for area recognition provided by this application. For each game character in the game screen image, different game characters have different skills, and when each skill is released, the skill special effects displayed by each released skill in the game screen image are also different. Therefore, if the image content in the game screen image is directly recognized to determine the image area where each game character is located by identifying each game character in the game screen image, this will lead to inaccurate recognition and extremely high recognition difficulty. This application mainly helps to identify the image area where each game character is located in the game screen image through the characteristic object that each game character has, and this characteristic object is the health bar that each game character has. As Figure 2a shown, first, a template image 100a containing a health bar can be prepared. As can be seen from Figure 2a , only a partial image of the health bar is included in the template image 100a. This is because, in different game states of each game character, the game level displayed in the health bar, the amount of health in the health bar, and the skill cooldown state are all different. Therefore, a partial image that is similar to the health bar images of each game character in different game states in most cases can be selected. Here, this partial image can refer to the partial image near the head of the health bar. The game level in the template image 100a can be any game level, such as any level from 1 to 15.

[0070] As Figure 2aAs shown, the image 101a is a game screen image, which includes three game characters, namely game character 101c, game character 103c, and game character 105c. Among them, the health bar of game character 101c is health bar 100c, the health bar of game character 103c is health bar 102c, and the health bar of game character 105c is health bar 104c. It can be seen that the health amounts, skill cooldown states, and game levels in health bar 100c, health bar 102c, and health bar 104c are different. The above template image 100a can be slid and matched in the image 101a, which can be understood as placing the template image 100a on top of the image 101a (as shown in interface 102a), and sequentially moving the placement position of the template image 100a in the image 101a. Each time it is moved to a placement position, the template image 100a is matched with the image in the image 101a that the template image 100a covers and obscures the image 101a (this image can be called the covered area image) for image similarity. Through this sliding matching method, the image area similar to the template image 100a in the image 101a can be obtained. As Figure 2a As shown, through the above sliding matching method, the image areas similar to the template image 100a in the image 101a are area 100b, area 101b, and area 102b (as shown in interface 103a).

[0071] Since the health bar of each game character usually has the same length and the same width, and these length and width are usually fixed values, therefore, the length and width of the health bar can be obtained. As shown in interface 104a, the position of the upper left corner in the image area 100b can be used as a reference point, and then according to the length and width of the health bar, the area 103b where the health bar of the recognized game character 101c is located can be obtained. The area length of the area 103b where the health bar is located is the length of the health bar, the area width is the width of the health bar, and the upper left corner of the area is the reference point corresponding to the image area 100b. The position of the upper left corner in the image area 101b can be used as a reference point, and then according to the length and width of the health bar, the area 105b where the health bar of the recognized game character 103c is located can be obtained. The area length of the area 105b where the health bar is located is the length of the health bar, the area width is the width of the health bar, and the upper left corner of the area is the reference point corresponding to the image area 101b. The position of the upper left corner in the image area 102b can be used as a reference point, and then according to the length and width of the health bar, the area 104b where the health bar of the recognized game character 105c is located can be obtained. The area length of the area 104b where the health bar is located is the length of the health bar, the area width is the width of the health bar, and the upper left corner of the area is the reference point corresponding to the image area 102b.

[0072] Since the position of each game character and its health bar is usually an up-and-down relationship, that is, the health bar of each game character is directly above each game character, the length of the area where the recognized health bar is located can be used as the length (horizontal length) of the area where the corresponding game character is located. In addition, since in this application, the area of the image where the game character is located in Image 101a is located by the area of the image where the health bar of the game character is located in Image 101a, different game characters are not distinguished. Then a height of a game character can be set, and each game character corresponds to the same height. As shown in Interface 105a, the lower edge of the area of the health bar area 103b can be used as the upper edge of the area where the corresponding game character is located. Then, by obtaining the height of the game character, the area 106b where the game character 101c is located in Image 101a can be obtained, and the area height of area 106b is the height of the game character. The lower edge of the area of the health bar area 104b can be used as the upper edge of the area where the corresponding game character is located. Then, by obtaining the height of the game character, the area 108b where the game character 105c is located in Image 101a can be obtained, and the area height of area 108b is the height of the game character. The lower edge of the area of the health bar area 105b can be used as the upper edge of the area where the corresponding game character is located. Then, by obtaining the height of the game character, the area 107b where the game character 103c is located in Image 101a can be obtained, and the area height of area 107b is the height of the game character.

[0073] The area where the health bar is located, 103b, and the area where the game character is located, 106b, can be merged to obtain the masked image area 1109b corresponding to the game character 101c. The area where the health bar is located, 105b, and the area where the game character is located, 107b, can be merged to obtain the masked image area 110b corresponding to the game character 103c. The area where the health bar is located, 104b, and the area where the game character is located, 108b, can be merged to obtain the masked image area 111b corresponding to the game character 105c. Therefore, the masked image areas in the final obtained Image 101a include the masked image area 109b, the masked image area 110b, and the masked image area 111b. The above Image 101a and the masked image areas in Image 101a (including the size and position of the masked image areas) can be sent to the terminal device 200b. After the terminal device 200b obtains Image 101a and the masked image areas in Image 101a, it can display Image 101a and the obtained bullet screens in the viewer client. The bullet screens are not displayed in the masked image areas in Image 101a and are normally displayed in the image areas of Image 101a other than the masked image areas. Please refer to Figure 2b, is a schematic diagram of a terminal page provided by the present application. The terminal page 107a is a page where the viewer client in the terminal device 200b displays the image 101a and the bullet screen. In order to facilitate the description of the method provided by the present application, the terminal page 107a displays the Figure 2a . As can be seen from the terminal page 107a, the bullet comments (including "Hurry up and use your big move", "666666666", "It's over", "The first assassin in the country", "Come on", "The house manager is here", "The assistant can't keep up" and "233333") are not displayed in the mask image area 109b, the mask image area 110b and the mask image area 111b, but are normally displayed in the image areas of the non-mask image area 109b, the non-mask image area 110b and the non-mask image area 111b. In fact, the image 101b and the bullet comments displayed by the audience client in the terminal device 200b are what are displayed in the terminal page 108a.

[0074] In the game, the game character generally has a blood bar on his head, and the blood bar information is fixed except for the amount of blood that will change. Therefore, in this embodiment, the position of the game character is located by locating the blood bar position of the game character. Therefore, the target detection problem can be simplified into an image matching problem, and the bullet screen mask area is obtained by image matching, and finally an unobstructed video bullet screen is generated on the user side. Detecting the position of the game character in this way is not only accurate, but also fast, and can meet real-time requirements. Through the method provided by the present application, since the image area where each game character is located in the game screen image is obtained by the blood bar associated with each game character, therefore, there is no need to identify the image area where each game character in the game screen image is located by image recognition, and it is only necessary to slide and match the template image 100a in the game screen image (that is, only the similarity between the template image and the coverage area image needs to be compared), which improves the efficiency of obtaining the image area where each game character is located in the game screen image. Therefore, the mask image area in each frame of the game screen image can be obtained at a faster speed, so that the audience client can also obtain each frame of the game screen image and the mask image area in each frame of the game screen image faster, and realize real-time display of each frame of the game screen image (the display method is to hide the barrage in the mask image area).

[0075] See also Figure 3 , is a flowchart of an image data processing method provided by the present application, such as Figure 3 As shown, the method may include:

[0076] Step S101: Obtain the first object image area where the virtual user object is located in the image; the first object image area is obtained based on the similar virtual attribute objects corresponding to the virtual user object cluster; the virtual user object cluster includes virtual user objects; there is a relative positional relationship between the virtual user object and the similar virtual attribute objects in the image.

[0077] Specifically, the present application can be applied to the game live broadcast scenario. The user client (referring to the client that can watch the game live broadcast, which can be a browser or a video player) can obtain the first object image area where the virtual user object is located in the image. Among them, the image can be any frame of the game screen image obtained during the game live broadcast process, and the virtual user object refers to the game character in the game application, that is, the game character. It can be seen that the virtual user object cluster is the cluster composed of all game characters in the game. Since during the game process, each virtual user object has its corresponding health bar, which represents the vital signs of each virtual user object during the game process. Therefore, the similar virtual attribute objects corresponding to the virtual user object cluster can refer to the health bar objects that each virtual user object has. In different game states, the display forms of the health bar objects corresponding to each virtual user object will also be different (for example, the blood volume in the health bar object will be different, and the skill cooling state in the health bar object will be different).

[0078] Since, in the actual game live streaming application scenario, there are several virtual user objects in a game application, and the skills possessed by each virtual user object are different. When each skill is released, the skill special effects displayed are also different. Therefore, if the virtual user object in the image is directly recognized through image recognition, and then the image area where the virtual user object is located in the image is obtained through the recognition result, it will be inaccurate and the recognition difficulty is extremely high. Therefore, the method provided in this application helps to obtain the image area where the virtual user object is located in the image through the similar virtual attribute object (i.e., the health bar object) that each virtual user object possesses. The reason is that usually in the game screen of a game application, a virtual user object corresponds to a similar virtual attribute object, and the virtual user object and its similar virtual attribute object are displayed simultaneously in the game screen. There is a relative position relationship between a certain virtual user object and its corresponding similar virtual attribute object, and this relative position relationship can represent the relative position between the virtual user object and its similar virtual attribute object. This relative position can include the regional relative position between the image area where the virtual user object is located and the image area where the corresponding similar virtual attribute object is located. For example, the relative position between the corresponding vertices (the image area can be a rectangular area, so it can include the vertices at the lower left corner of the area, the vertices at the upper left corner of the area, the vertices at the lower right corner of the area, and the vertices at the upper right corner of the area) of the two image areas. The relative position relationship between each virtual user object in the virtual user cluster and its corresponding similar virtual attribute object is the same. Therefore, it can be understood that there is no need to distinguish different virtual user objects. After detecting the image area where the similar virtual attribute object is located in the image, the image area where the similar virtual attribute object is located can be combined with the relative position between the similar virtual attribute object and the virtual user object to obtain the image area where the corresponding virtual user object is located in the image.

[0079] The image area where the virtual user object is located in the image can be called the first object image area, and the image area where the similar virtual attribute object is located in the image can be called the second object image area. In this application, it mainly describes how to determine the image area where the virtual user object in an image is located. It can be understood that during the game live streaming process, several frames of images in the game live streaming screen can be obtained, and the process of identifying the image area where the virtual user object is located in each frame of the obtained image is the same and independent of each other.

[0080] Among them, each frame of the image in which the virtual user object in the present application needs to be recognized (i.e., the first object image region) can be obtained by the live client. Herein, the live client refers to the client used to support the host for game live broadcast. During the game live broadcast by the host, the live client can obtain each frame of game screen image during the game live broadcast process of the host. The following describes the first object image region obtained by the user client (including the region size and region position of the first object image region in the image).

[0081] Optionally, the first object image region obtained by the user client can be the one obtained by the above live client by recognizing the image region where the virtual user object is located in the image, or the one obtained by the server by recognizing the image region where the virtual user object is located in the image, or the one obtained by the user client itself by recognizing the image region where the virtual user object is located in the image. That is, the live client, the server, and the user client can all be the execution entities for obtaining the first object image region in the image, and the principles of how the live client, the server, and the user client recognize and obtain the first object image region in the image are the same (i.e., the recognition processes are the same).

[0082] If the above first object image region is obtained by the live client by recognizing the image region where the virtual user object is located in the image, the process for the user client to obtain the first object image region can be as follows:

[0083] Support the host to conduct game live broadcast through the live client. During the game live broadcast process, the live client can obtain each frame of game screen image in the game screen when the host conducts the game live broadcast. This game screen image can be the image where the above first object image region is located. The live client can recognize the image region where the virtual user object is located in the image (for the specific recognition process, see the following), and obtain the first object image region of the virtual user object in the image. The live client can send the obtained first object image region and the image corresponding to this first object image region to the server, and the server can forward this first object image region and the corresponding frame of the image to the user client. In other words, in this scenario, the first object image region obtained by the user client and the image where this first object image region is located are first sent by the live client to the server and then forwarded by the server to itself.

[0084] If the above first object image region is obtained by the server by recognizing the image region where the virtual user object is located in the image, the process for the user client to obtain the first object image region can be as follows:

[0085] First, during the game live broadcast by the host, the live broadcast client can obtain each frame of the game screen during the game live broadcast, and each frame of the game screen corresponds to an image. The live broadcast client can send each obtained image to the server, and the server can identify the image area where the virtual user object is located in the image and obtain the first object image area where the virtual user object is located in the image. The server can send the obtained first object image area and the image where the first object image area is located to the user client.

[0086] If the above-mentioned first object image area is obtained by the user client itself by identifying the image area where the virtual user object is located in the image, the process for the user client to obtain the first object image area can be as follows:

[0087] First, during the game live broadcast by the host, the live broadcast client can obtain each frame of the image corresponding to each frame of the game screen during the game live broadcast. The live broadcast client can send each obtained frame of the image to the server, and the server can send each obtained frame of the image to the user client again. After obtaining each frame of the image, the user client can identify by itself and obtain the first object image area where the virtual user object is located in the image.

[0088] It should be noted that in the actual application scenario, usually the live broadcast client or the server is used to identify and obtain the first object image area where the virtual user object is located in the image. However, in some scenarios, the user client can also implement the identification of the image to obtain the first object image area of the virtual user object in the image.

[0089] Taking the user client as the execution subject below, a specific description is given on how to obtain the first object image area where the virtual user object is located in the image. It can be understood that if the live broadcast client or the server is used to identify the first object image area where the virtual user object is located in the image, the identification process here is the same as the following process:

[0090] Since the method provided in this application indirectly locates the image area (i.e., the first object image area) where the virtual user object is located in the image through the similar virtual attribute objects corresponding to the virtual user object cluster, it is first necessary to identify the image area where the similar virtual attribute objects are located in the image. In the method provided in this application, the similar virtual attribute object used is the health bar object that each virtual user object has. In some scenarios, the similar virtual attribute object can also be other objects associated with the virtual user object, and one virtual user object corresponds to one similar virtual attribute object. Since the relative positions of the similar attribute objects and the corresponding virtual user objects in the image have certain rules (such as how much the distance between them is, the layout rules between them in the up, down, left, and right directions, etc.), the image area where the corresponding virtual user object is located in the image can be deduced from the image area where the similar attribute objects are located in the image. The following is the process of determining the second object image area where the similar virtual attribute object is located in the image:

[0091] First, a matching image containing the similar virtual attribute object can be obtained. The matching image can contain some of the similar virtual attribute objects or all of the similar virtual attribute objects. In this application, the similar virtual attribute object is the health bar object. Since during the game live broadcast, the display form of the health bar object corresponding to the virtual user object may change dynamically in real time, such as changes in the amount of health displayed in the health bar object, changes in the character game level displayed in the health bar object, and changes in the skill cooling state displayed in the health bar object, etc. Therefore, it is best for the matching image to contain the similar partial objects that the health bar objects in various display forms have during the game live broadcast. Optionally, an image containing partial objects near the head of the health bar object can be used as the matching image (such as the template image 100a in the above Figure 2a ), and the matching image can contain the head of the health bar object (i.e., the partial object containing the character game level) and 2 grids of health near the head of the health bar object (for example, there are 10 grids of health in the health bar, but only 2 grids of health near the head of the health bar are taken, or it can be 0 grids of health or 1 grid of health, etc., which is specifically determined according to the actual application scenario and is not limited thereto).

[0092] The matching image can be slid and traversed on the image obtained during the game live broadcast. The sliding step size for the sliding traversal can be obtained. According to this sliding step size, the matching image can be slid and traversed on the image. The process of the sliding traversal is also the process of image matching. For example, the sliding step size can be 1 pixel (it can also be 2 pixels, 3 pixels, etc., which is specifically determined according to the actual application scenario and is not restricted here). Then, this sliding traversal can be understood as placing the matching image on the image and sliding it successively, with the distance of each slide on the image being 1 pixel. In other words, the matching image slides once every 1 pixel distance on the image. The sliding position of the matching image in the image after the sliding traversal can be obtained. Each time the matching image slides on the image, it corresponds to a sliding position. The covered area image of the image where the matching image causes coverage and occlusion at each sliding position can be obtained. One sliding position corresponds to one covered area image. The matching image can be subjected to image similarity matching with each covered area image, that is, the similarity between the covered area image and the matching image can be matched, and the image matching degree can be obtained. One covered area image corresponds to one image matching degree. An image matching degree threshold can be set. The image area in the image of the covered area image with an image matching degree greater than or equal to the image matching degree threshold can be called the matching image area. There can be multiple matching image areas. The matching image areas can be recorded first. The covered area image corresponding to the matching image area is the local image in the matched image that has image similarity with the matching image. In other words, the covered area image corresponding to the matching image area has a high similarity (it can also be called the matching degree) with the matching image. Among them, the algorithm for matching the image similarity between the matching image and the covered area image can be the matchTemplate (a template matching function) provided by OpenCV (a cross-platform computer vision library distributed under the Berkeley Software Distribution license).

[0093] When obtaining the above-mentioned matching image areas, there may be the following several situations:

[0094] First, since the scaling ratios corresponding to the matching image and the image (the game screen image for identifying the first object image area) may be different, this will cause the image areas of the similar virtual attribute objects in the matching image and the image not to fit when the matching image is placed on the image and slid through. In other words, this will cause even if the matching image slides to the position of the similar virtual attribute object in the image, due to the matching image being too large or too small, the covering area image corresponding to the matching image will also be too large or too small. If the covering area image is too large, it means that in addition to including the similar virtual attribute object, the covering area image also includes some background images of the similar virtual attribute object in the image. If the covering area image is too small, it means that the covering area image only includes a very small part of the object content of the similar virtual attribute object. These will all result in inaccurate matching results when performing similarity matching between the matching image and the covering area image. Therefore, it is necessary to unify the scaling ratio of the matching image and the scaling ratio of the image to be the same. The following describes how to unify the scaling ratio of the matching image and the scaling ratio of the image to be the same:

[0095] The scaling ratio corresponding to the image can be called the first scaling ratio, and the scaling ratio corresponding to the matching image can be called the second scaling ratio. When the first scaling ratio and the second scaling ratio are not equal, a unified scaling ratio can be obtained, and this unified scaling ratio can be any scaling ratio (for example, 50%). The matching image can be scaled through this unified scaling ratio to obtain a scaled matching image. The image can be scaled through this unified scaling ratio to obtain a scaled image. Furthermore, the scaled matching image can be slid through the scaled image to obtain the above-mentioned matching image area.

[0096] Second, since the colors of the similar virtual attribute objects possessed by different virtual user objects may be different, therefore, for the accuracy of recognition, it is necessary to grayscale the above-mentioned matching image and the image. By grayscaling the matching image and the image, the problem of different colors of the similar virtual attribute objects can be solved. By grayscaling the image, a grayscale image corresponding to the image can be obtained, and the grayscale image corresponding to the image can be called the first grayscale image. By grayscaling the matching image, a grayscale image corresponding to the matching image can be obtained, and the grayscale image corresponding to the matching image can be called the second grayscale image. The second grayscale image can be slid through the first grayscale image to obtain the above-mentioned matching image area.

[0097] The above two situations can be combined and used. Specifically: after obtaining the scaled image and the scaled matching image, the scaled image and the scaled matching image can be grayscaled, and then the grayscaled scaled matching image can be slid and matched on the grayscaled scaled image to obtain the above-mentioned matching image area.

[0098] The matching position corresponding to each matched image region in the image can be obtained. Generally, the matched image region is a rectangular region. Therefore, the pixel position of the pixel point at the upper left corner of the matched image region in the image can be used as the matching position corresponding to the matched image region. Assume that the obtained matched image regions include a first image matching region and a second image matching region. Since there can be multiple obtained matched image regions, the first image matching region and the second image matching region can be any two of the multiple matched image regions. Through the above method, the matching position corresponding to the first image matching region in the image can be obtained, and the matching position corresponding to the first image matching region can be called the first matching position. Similarly, the matching position corresponding to the second image matching region in the image can be obtained, and the matching position corresponding to the second image matching region can be called the second matching position.

[0099] A distance threshold can be set (for example, 1 pixel distance or 2 pixel distances, etc.). When the position distance between the above first matching position and the second matching position is greater than the set distance threshold, it indicates that the similar virtual attribute objects pointed to by the first image matching region corresponding to the first matching position and the second image matching region corresponding to the second matching position are the same. Therefore, in this case, the first matching position and the second matching position need to be merged. The merging method is to select (it can be arbitrarily selected) one of the first matching position and the second matching position as the target position. After one merging, the merged matching position will not be merged with other matching positions. When the position distance between the above first matching position and the second matching position is greater than the set distance threshold, there is no need to merge the first matching position and the second matching position, and both the first matching position and the second matching position can be used as target positions. Through this method, the matching positions corresponding to each image matching region can be merged, and finally one or more target positions are obtained. One target position corresponds to one similar virtual attribute object, that is, one target position indicates that 1 blood bar object is recognized in the image, and multiple target positions indicate that multiple blood bar objects are recognized in the image. The sizes of the matched image, the covered area image, and the image matching region are the same, all being the size of the matched image.

[0100] Please refer to Figure 4 , which is a schematic diagram of a scenario for obtaining the target position provided by this application. As Figure 4As shown, the image 100d is a frame of a game screen image. The image 100d includes a virtual user object 101j, a virtual user object 103j, and a virtual user object 105j. Among them, the similar virtual attribute object corresponding to the virtual user object 101j is the virtual attribute object 100j, the similar virtual attribute object corresponding to the virtual user object 103j is the virtual attribute object 102j, and the similar virtual attribute object corresponding to the virtual user object 105j is the virtual attribute object 104j. The amount of blood and the skill cooldown status in the virtual attribute object 100j, the virtual attribute object 102j, and the virtual attribute object 104j are all different. The matching image 106j can be slid on the image 100d for matching, and image matching regions 100e, 101e, 102e, 108e, 109e, and 111e can be obtained. Among them, usually when the matching image is slid on the image for matching, multiple image matching regions (such as 3 or 5, etc.) will be obtained for a similar virtual attribute object in the image. Here, as Figure 4 shown, the above-mentioned image matching regions 100e, 101e, and 102e correspond to the virtual attribute object 100j; the image matching regions 108e and 109e correspond to the virtual attribute object 104j; the image matching region 111e corresponds to the virtual attribute object 102j.

[0101] Next, as shown in the interface 101d, the matching position 103e of the image matching region 100e (the matching position 103e is the position of the upper left corner of the region of the image matching region 100e), the matching position 104e of the image matching region 101e (the matching position 104e is the position of the upper left corner of the region of the image matching region 101e), the matching position 105e of the image matching region 102e (the matching position 105e is the position of the upper left corner of the region of the image matching region 102e), the matching position 106e of the image matching region 108e (the matching position 106e is the position of the upper left corner of the region of the image matching region 108e), the matching position 107e of the image matching region 109e (the matching position 107e is the position of the upper left corner of the region of the image matching region 109e), and the matching position 110e of the image matching region 111e (the matching position 110e is the position of the upper left corner of the region of the image matching region 111e) can be obtained

[0102] As shown in interface 102d, since the position distance between matching positions 103e, 104e, and 105e (referring to the two matching positions with the smallest position distance) is less than the distance threshold, the matching positions 103e, 104e, and 105e can be merged (the principle of merging is to select one of the matching positions 103e, 104e, and 105e as the target position), and the matching position 104e can be used as the target position m1. Usually, when merging multiple matching positions, the matching position in the middle of these multiple matching positions is selected. Optionally, a matching position can also be randomly selected from the multiple matching positions that need to be merged as the target position. It can be understood that the errors between the obtained multiple matching positions are offset around a certain position, so selecting the matching position at the center is usually more accurate. Since the position distance between matching positions 106e and 107e is less than the distance threshold, the matching positions 106e and 107e can be merged (similarly, the principle of merging is to randomly select one of the matching positions 106e and 107e as the target position), and the matching position 106e can be used as the target position m2. There is no matching position near the matching position 110e with a position distance less than the distance threshold, so the matching position 110e can be directly used as the target position m3.

[0103] Through the above process, 3 target positions in the image 100d are obtained (as shown in interface 103d), and these 3 target positions are the target position m1, the target position m2, and the target position m3 respectively. Among them, there is Figure 4 It can be seen that the target position 1 corresponds to the upper left corner position of the virtual attribute object 100j, the target position 2 corresponds to the upper left corner position of the virtual attribute object 104j, and the target position 3 corresponds to the upper left corner position of the virtual attribute object 102j.

[0104] The second object image region can be obtained based on the obtained target position: In this application, the length and width of the similar virtual attribute object (i.e., the health bar object) possessed by each virtual user object are usually fixed. Therefore, the object length and object width corresponding to the similar virtual attribute object can be preset in advance. Subsequently, the object length and object width corresponding to the similar virtual attribute object can be directly obtained. Generally, the obtained second object image region is also a rectangular region. Therefore, the target position can be used as the position of the upper left corner of the region of the second object image region. Moreover, the region length of the second object image region is the object length of the obtained similar virtual attribute object, and the region width of the second object image region is the object width of the obtained similar virtual attribute object. Through the above method, the second object image region in the image can be obtained, and the obtained second object image region is the image region where the recognized similar virtual attribute object is located in the image. When there are multiple target positions, each target position corresponds to a second object image region, that is, one target position corresponds to one second object image region.

[0105] After obtaining the second object image region, the position image region can be obtained through this second object image region. One second object image region corresponds to one position image region. In other words, one target position corresponds to one position image region. The position image region refers to the image region where each virtual user object is located in the image, and one virtual user object corresponds to one position image region. If there is only one position image region (that is, only one target position / second object image region), then this position image region can be directly used as the above-mentioned first object image region. If there are multiple position image regions, then these multiple position image regions can be merged to obtain the first object image region. The following is the process of how to obtain the corresponding position image region through one second object image region. The process of obtaining the corresponding position image region for each second object image region is the same. The first object image region can only include the position image region and not include the second object image region, or the first object image region can also include both the position image region and the second object image region. If the first object image region can only include the position image region, then the position image region or the merged position image region can be directly used as the first object image region. If the first object image region includes both the position image region and the second object image region, then all the position image regions and all the second object image regions can be merged to obtain the first object image region.

[0106] First, the object height and object length of the virtual user object can be obtained. The position image area is also a rectangular area. The area length of the position image area is the object length of the virtual user object, and the area height of the position image area is the object height of the virtual user object. Since in this application, the image area where the virtual user object is located in the image is not directly obtained by recognizing the virtual user object in the image, but by quickly obtaining the image area where the virtual user object is located in the image by means of a similar virtual attribute object. Therefore, in this application, different virtual user objects are not distinguished, and the object height corresponding to each virtual user object can be set to an object height, that is, the area heights of the first object image areas corresponding to all virtual user objects are the same, and this area height is the object height corresponding to each virtual user object set (for example, 100 pixel heights).

[0107] Among them, in the game live broadcast scenario of this application, the relative positions of the virtual user object (i.e., the game character / hero) and the similar virtual attribute object (i.e., the health bar object) in the game screen image are usually in a vertically displayed up-and-down position relationship, that is, the corresponding similar virtual attribute object is displayed directly above the virtual user object. Therefore, the edge position of the lower edge of the second object image area in the vertical direction can be used as the edge position of the upper edge of the position image area in the vertical direction. The object lengths corresponding to each virtual user object can also be the same, and the area length (the length in the horizontal direction) of the second object image area can be directly used as the object length of the virtual user object. Or, after appropriately shortening or increasing the area length of the first object image area, it can be used as the object length of the virtual user object. The position of the upper edge of the position image area can be determined by the position of the center point of the lower edge of the second object image area, so that the position of the center point of the upper edge of the position image area is the same as the position of the center point of the lower edge of the second object image area, that is, the final obtained position image area and the corresponding second object image area are in a vertically up-and-down and horizontally centered position relationship with each other.

[0108] Therefore, through the above process, the first object image area can be obtained according to the area position of the second object image area in the image, the relative position of the virtual user object and the second object image area in the image, the object length of the virtual user object, and the object height of the virtual user object.

[0109] Please refer to Figure 5 , which is a schematic diagram of a scenario for obtaining an object image area provided by this application. Interface 103d is the above-mentioned Figure 4The interface 103d in it includes target position 1, target position 2, and target position 3. The object length (assumed to be length 1) and object width (assumed to be width 1) of a similar virtual attribute object can be obtained. Therefore, as shown in interface 104d, the second object image area 100f can be obtained based on target position m1, length 1, and width 1, the second object image area 102f can be obtained based on target position m2, length 1, and width 1, and the second object image area 101f can be obtained based on target position m3, length 1, and width 1. Then the object height (assumed to be height 1) of the virtual user object can be obtained. Here, the area length of the second object image area is directly used as the object length of the virtual user object. As shown in interface 105d, the position image area 103f corresponding to the second object image area 100f can be obtained. The upper edge of the area of the position image area 103f is the lower edge of the area of the second object image area 100f. The position image area 103f is the position image area corresponding to target position m1. The position image area 104f corresponding to the second object image area 102f can be obtained. The upper edge of the area of the position image area 104f is the lower edge of the area of the second object image area 102f. The position image area 104f is the position image area corresponding to target position m2. The position image area 105f corresponding to the second object image area 101f can be obtained. The upper edge of the area of the position image area 105f is the lower edge of the area of the second object image area 101f. The position image area 105f is the position image area corresponding to target position m3.

[0110] Here, the first object image area includes both the position image area and the second object image area. Therefore, as shown in interface 106d, the above-mentioned second object image area 100f and the position image area 103f are merged to obtain area 106f; the above-mentioned second object image area 102f and the position image area 104f are merged to obtain area 107f; the above-mentioned second object image area 101f and the position image area 105f are merged to obtain area 108f. The above-mentioned areas 106f, 107f, and 108f are merged by area, and the first object image area where the virtual user object is located in image 100d can be obtained. Among them, please also refer to the above Figure 4 , area 106f is the image area where the virtual user object 101j and its corresponding similar virtual attribute object 100j are located in image 100d, area 107f is the image area where the virtual user object 105j and its corresponding similar virtual attribute object 104j are located in image 100d, and area 108f is the image area where the virtual user object 103j and its corresponding similar virtual attribute object 102j are located in image 100d.

[0111] Please refer to Figure 6 , which is a schematic diagram of a scenario for region merging provided by this application. As Figure 6 shown, in the image 100g, there are virtual user objects 102h and virtual user object 103h. Among them, the similar virtual attribute object corresponding to the virtual user object 102h is 100h, and the similar virtual attribute object corresponding to the virtual user object 103h is 101h. As shown in the interface 101g, the image area where the virtual user object 102h is located in the image 100g can be the area 104h (a rectangular area that includes the similar virtual attribute object 103h), and the image area where the virtual user object 103h is located in the image 100g can be the area 105h (a rectangular area that includes the similar virtual attribute object 101h). The area 104h and the area 105h have an intersecting image area. Therefore, the image area where the virtual user object is located in the image 100g (i.e., the first object image area) can be the area 106h obtained by merging the area 104h and the area 105h (as shown in the interface 102g). The area 106h is the total image area occupied by the area 104h and the area 105h in the image 100g.

[0112] Step S102, obtain multimedia data for independent display on an image;

[0113] Specifically, the user client can obtain multimedia data for independent display on an image, and this multimedia data can be the bullet screen data entered by netizens during a game live broadcast. Among them, the bullet screen data refers to commentative remarks. In a certain scenario, the data form of the bullet screen data can be in the form of text, or in the form of an image, or in the form of a video, etc., and there is no limitation on this.

[0114] Step S103, perform hidden display on the multimedia data in the first object image area, and perform superimposed display on the multimedia data in the background image area and the background image area; the background image area is the image area in the image other than the first object image area;

[0115] Specifically, the user client can set the obtained first object image area as a mask image area, where the mask image area refers to the protected image area, and the multimedia data is not displayed in the mask image area. The user client can be carried on a terminal device. The user client can display the obtained image through the terminal device and can independently display the above multimedia data on the displayed image. The multimedia data can be displayed in a flowing manner on the image, for example, flowing from left to right on the terminal page of the terminal device. Moreover, when the multimedia data is displayed on the image, the multimedia data in the mask image area in the image is not displayed (which can be called hidden display), and the multimedia data in the background image area of the image is superimposed and displayed with the background image area (that is, the multimedia data is superimposed and displayed on the background image area). Among them, the background image area is the image area in the image except for the first object image area. In this way, the multimedia data (i.e., bullet screen data) can be displayed without blocking the virtual user object (i.e., game character) in the game live broadcast screen.

[0116] Optionally, the method of hidden display of the multimedia data in the first object image area can also be to display the multimedia data in the first object image area according to a certain transparency, or to reduce the multimedia data in the first object image area to a certain size for display. How the multimedia data in the first object image area is displayed can be determined according to the actual application scenario, and there is no limit to this. A display button can be provided in the user client, and the user client can respond to the click operation on the display button (which can be called the display operation for the multimedia data), and then display the multimedia data in the image.

[0117] Please refer to Figure 7 , which is another schematic diagram of the page of the terminal provided by this application. As Figure 7 shown, the user client can display the image 100d in the above Figure 4 through the terminal device 100g carrying it, and can also display bullet screen data (i.e., multimedia data, including "hahaha", "laughing to death", "this operation is so awesome", "no cooperation among team members", "666666666666666", and "23333") on the image 100d. Since through the above Figure 5The corresponding embodiment has obtained the first object image area where the virtual user object is located in the image 100d (including area 106f, area 107f, and area 108f). Therefore, the user client can set this first object image area as the mask image area. As shown in the terminal page 108f, the bullet screen data is not displayed in this mask image area (for ease of description, the area border of the first object image area is shown in the terminal page 108f). In fact, when the terminal device 100g displays the image 100d and the bullet screen data, what is displayed is the picture shown in the terminal page 109f.

[0118] This application obtains the first object image area where the virtual user object is located in the image; the first object image area is obtained according to the similar virtual attribute objects corresponding to the virtual user object cluster; the virtual user object cluster includes the virtual user object; the virtual user object and the similar virtual attribute objects have a relative position relationship in the image; obtains the multimedia data for independent display on the image; hides and displays the multimedia data in the first object image area, and superimposes and displays the multimedia data in the background image area with the background image area; the background image area is the image area in the image except the first object image area. It can be seen that the method proposed in this application can identify the image area (i.e., the first object image area) where the virtual user object is located in the image through the similar virtual attribute objects of the virtual user object, so as to realize that when the multimedia data is displayed in the image, the multimedia data will not block the virtual user object in the image.

[0119] Please refer to Figure 8 , which is a schematic structural diagram of an image data processing device provided by this application. The image data processing device can be a computer program (including program code) running in a computer device. For example, the image data processing device is an application software, and the image data processing device can be used to execute the corresponding steps in the method provided by the embodiments of this application. As Figure 8 shown, the image data processing device 1 may include: an identification module 11, an acquisition module 12, and a display module 13;

[0120] The identification module 11 is used to obtain the first object image area where the virtual user object is located in the image; the first object image area is obtained according to the similar virtual attribute objects corresponding to the virtual user object cluster; the virtual user object cluster includes the virtual user object; the virtual user object and the similar virtual attribute objects have a relative position relationship in the image;

[0121] The acquisition module 12 is used to obtain the multimedia data for independent display on the image;

[0122] A display module 13, configured to perform hidden display on multimedia data in a first object image area and perform superimposed display on the multimedia data in a background image area and the background image area; the background image area is the image area in the image except the first object image area.

[0123] For the specific functional implementation manners of the recognition module 11, the acquisition module 12, and the display module 13, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0124] The recognition module 11 includes: a matching acquisition unit 111, a traversal unit 112, and a region determination unit 113;

[0125] The matching acquisition unit 111 is configured to acquire a matching image including a similar virtual attribute object;

[0126] The traversal unit 112 is configured to slide and traverse the matching image in the image to obtain a matching image area corresponding to the matching image in the image; the matching image area and the matching image have image similarity;

[0127] The region determination unit 113 is configured to determine the first object image area according to the matching image area.

[0128] For the specific functional implementation manners of the matching acquisition unit 111, the traversal unit 112, and the region determination unit 113, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0129] The traversal unit 112 includes: a ratio acquisition subunit 11201, a ratio determination subunit 11202, a scaling subunit 11203, and a first traversal subunit 11204;

[0130] The ratio acquisition subunit 11201 is configured to acquire a first scaling ratio corresponding to the image and a second scaling ratio corresponding to the matching image;

[0131] The ratio determination subunit 11202 is configured to acquire a unified scaling ratio if the first scaling ratio is not equal to the second scaling ratio;

[0132] The scaling subunit 11203 is configured to scale the image and the matching image according to the unified scaling ratio to obtain a scaled image and a scaled matching image;

[0133] The first traversal subunit 11204 is configured to slide and traverse the scaled matching image on the scaled image to obtain the matching image area.

[0134] Among them, for the specific functional implementation manners of the ratio acquisition subunit 11201, the ratio determination subunit 11202, the scaling subunit 11203, and the first traversal subunit 11204, please refer to Figure 3 step S101 in the corresponding embodiment, which will not be elaborated here.

[0135] Among them, the traversal unit 112 includes: a step size acquisition subunit 11205, a coverage acquisition subunit 11206, a similarity matching subunit 11207, and a first region determination subunit 11208;

[0136] The step size acquisition subunit 11205 is configured to acquire a sliding step size, and slide and traverse the matching image in the image based on the sliding step size;

[0137] The coverage acquisition subunit 11206 is configured to acquire a sliding position after the matching image is slid and traversed, and acquire a coverage area image corresponding to the matching image at the sliding position in the image;

[0138] The similarity matching subunit 11207 is configured to perform image similarity matching on the coverage area image and the matching image to obtain an image matching degree;

[0139] The first region determination subunit 11208 is configured to, when the image matching degree is greater than or equal to an image matching degree threshold, determine the image region pointed to by the coverage area image in the image as a matching image region.

[0140] Among them, for the specific functional implementation manners of the step size acquisition subunit 11205, the coverage acquisition subunit 11206, the similarity matching subunit 11207, and the first region determination subunit 11208, please refer to Figure 3 step S101 in the corresponding embodiment, which will not be elaborated here.

[0141] Among them, the traversal unit 112 includes: a grayscale image acquisition subunit 11209 and a second traversal subunit 11210;

[0142] The grayscale image acquisition subunit 11209 is configured to acquire a first grayscale image corresponding to the image and a second grayscale image corresponding to the matching image;

[0143] The second traversal subunit 11210 is configured to slide and traverse the second grayscale image in the first grayscale image to obtain a matching image region corresponding to the matching image.

[0144] Among them, for the specific functional implementation manners of the grayscale image acquisition subunit 11209 and the second traversal subunit 11210, please refer to Figure 3 step S101 in the corresponding embodiment, which will not be elaborated here.

[0145] Among them, the virtual user object is a game character object; the similar virtual attribute object is the health bar object corresponding to the game character object; the matching image is an image containing the health bar object.

[0146] Among them, the matching image area includes a first image matching area and a second image matching area;

[0147] The area determination unit 113 includes: a position acquisition subunit 1131, a first target determination subunit 1132, and a second target determination subunit 1133;

[0148] The position acquisition subunit 1131 is configured to acquire the first matching position of the first matching image area in the image and the second matching position of the second matching image area in the image;

[0149] The first target determination subunit 1132 is configured to, when the position distance between the first matching position and the second matching position is less than or equal to the distance threshold, determine the first matching position or the second matching position as the target position, and determine the first object image area according to the target position;

[0150] The second target determination subunit 1133 is configured to, when the position distance between the first matching position and the second matching position is greater than the distance threshold, determine both the first matching position and the second matching position as the target position, and determine the first object image area according to the target position.

[0151] Among them, for the specific functional implementation manners of the position acquisition subunit 1131, the first target determination subunit 1132, and the second target determination subunit 1133, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0152] Among them, the first target determination subunit 1132 includes: a region acquisition subunit 11321 and a region merging subunit 11322;

[0153] The region acquisition subunit 11321 is configured to, when there are at least two target positions, acquire the position image regions corresponding to each target position in the image; the position image region is the region where the virtual user object is located;

[0154] The region merging subunit 11322 is configured to merge the position image regions corresponding to each target position to obtain the first object image area.

[0155] Among them, for the specific functional implementation manners of the region acquisition subunit 11321 and the region merging subunit 11322, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0156] Among them, the first target determination subunit 1132 includes: a first acquisition subunit 11323, a second region determination subunit 11324, and a third region determination subunit 11325;

[0157] The first acquisition subunit 11323 is configured to acquire the object width of the similar virtual attribute object and the object length of the similar virtual attribute object;

[0158] The second region determination subunit 11324 is configured to determine a second object image region of the similar virtual attribute object in the image according to the target position, the object width of the similar virtual attribute object, and the object length of the similar virtual attribute object;

[0159] The third region determination subunit 11325 is configured to determine a first object image region according to the second object image region and the virtual user object.

[0160] Among them, for the specific functional implementation manners of the first acquisition subunit 11323, the second region determination subunit 11324, and the third region determination subunit 11325, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0161] Among them, the third region determination subunit 11325 includes: a second acquisition subunit 113251 and a fourth region determination subunit 113252;

[0162] The second acquisition subunit 113251 is configured to acquire the object height of the virtual user object and acquire the object length of the virtual user object;

[0163] The fourth region determination subunit 113252 is configured to determine the first object image region according to the object height of the virtual user object, the object length of the virtual user object, the region position of the second object image region in the image, and the relative position of the virtual user object and the second object image region in the image.

[0164] Among them, for the specific functional implementation manners of the second acquisition subunit 113251 and the fourth region determination subunit 113252, please refer to Figure 3 Step S101 in the corresponding embodiment, which will not be elaborated here.

[0165] Among them, the display module 13 includes: a mask setting unit 131 and a response unit 132;

[0166] The mask setting unit 131 is configured to set the first object image region as a mask image region;

[0167] The hidden display unit 132 is configured to perform hidden display on the multimedia data in the first object image region based on the mask image region.

[0168] Among them, for the specific functional implementation manners of the mask setting unit 131 and the hidden display unit 132, please refer to Figure 3 step S103 in the corresponding embodiment, which will not be elaborated here.

[0169] This application obtains a first object image area where a virtual user object is located in an image; the first object image area is obtained according to similar virtual attribute objects corresponding to a virtual user object cluster; the virtual user object cluster includes virtual user objects; there is a relative position relationship between the virtual user object and the similar virtual attribute objects in the image; obtains multimedia data for independent display on the image; hides and displays the multimedia data in the first object image area, and superimposes and displays the multimedia data in the background image area and the background image area; the background image area is the image area in the image except the first object image area. It can be seen that the method proposed in this application can identify the image area (i.e., the first object image area) where the virtual user object is located in the image through the similar virtual attribute objects of the virtual user object, so as to realize that when displaying multimedia data in the image, the multimedia data will not block the virtual user object in the image.

[0170] Please refer to Figure 9 , which is a schematic structural diagram of a computer device provided by this application. As Figure 9 shown, the computer device 1000 may include: a processor 1001, a network interface 1004, and a memory 1005. In addition, the computer device 1000 may further include: a user interface 1003 and at least one communication bus 1002. Among them, the communication bus 1002 is used to implement connection communication between these components. Among them, the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 1005 may further be at least one storage device located far from the aforementioned processor 1001. As Figure 9 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0171] In Figure 9In the computer device 1000 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for users to input; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to implement the description of the image data processing method in the corresponding embodiment mentioned above Figure 3 in the corresponding embodiment. It should be understood that the computer device 1000 described in the present application can also execute the description of the image data processing device 1 in the corresponding embodiment mentioned above Figure 8 and will not be elaborated here. In addition, the description of the beneficial effects of the same method will not be elaborated either.

[0172] In addition, it should be pointed out here that: the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the image data processing device 1 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the image data processing method in the corresponding embodiment mentioned above Figure 3 and will not be elaborated here. In addition, the description of the beneficial effects of the same method will not be elaborated either. For the technical details not disclosed in the embodiment of the computer storage medium involved in the present application, please refer to the description of the method embodiment of the present application. As an example, the program instructions can be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. The multiple computer devices distributed at multiple locations and interconnected through a communication network can form a blockchain system.

[0173] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The above program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the above storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0174] The above-disclosed are only the preferred embodiments of the present application. Of course, the scope of the rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. An image data processing method, characterized in that, it includes: Obtaining a first object image area where a virtual user object is located in the image; The first object image area is obtained based on the detection of similar virtual attribute objects corresponding to a virtual user object cluster in the image. The virtual user object cluster includes the virtual user object. The first object image area is determined according to the positions of the detected similar virtual attribute objects in the image and the relative position relationship between the detected similar virtual attribute objects and the virtual user object. The similar virtual attribute object is a health bar object corresponding to the virtual user object cluster; Obtaining multimedia data for independent display on the image; Hiding and displaying the multimedia data in the first object image area, and superimposing and displaying the multimedia data in the background image area with the background image area; the background image area is the image area in the image other than the first object image area.

2. The method according to claim 1, characterized in that, The obtaining of the first object image area where the virtual user object is located in the image includes: Obtaining a matching image containing the similar virtual attribute object; Sliding and traversing the matching image in the image to obtain a matching image area corresponding to the matching image in the image; the matching image area has image similarity with the matching image; Determining the first object image area according to the matching image area.

3. The method according to claim 2, characterized in that, The sliding and traversing of the matching image in the image to obtain the matching image area corresponding to the matching image in the image includes: Obtaining a first scaling ratio corresponding to the image and a second scaling ratio corresponding to the matching image; If the first scaling ratio is not equal to the second scaling ratio, obtaining a unified scaling ratio; Scaling the image and the matching image according to the unified scaling ratio to obtain a scaled image and a scaled matching image; Sliding and traversing the scaled matching image on the scaled image to obtain the matching image area.

4. The method according to claim 2, characterized in that, The sliding and traversing of the matching image in the image to obtain the matching image area corresponding to the matching image in the image includes: Obtaining a sliding step size, and sliding and traversing the matching image in the image based on the sliding step size; Obtaining the sliding position after the matching image is slid and traversed, and at the sliding position in the image, obtaining a covered area image corresponding to the matching image; Performing image similarity matching on the covered area image and the matching image to obtain an image matching degree; When the image matching degree is greater than or equal to an image matching degree threshold, determining the image area pointed to by the covered area image in the image as the matching image area.

5. The method according to claim 2, characterized in that, Sliding and traversing the matching image in the image to obtain a matching image region corresponding to the matching image in the image, including: Obtaining a first grayscale image corresponding to the image, and obtaining a second grayscale image corresponding to the matching image; Sliding and traversing the second grayscale image in the first grayscale image to obtain the matching image region corresponding to the matching image.

6. The method according to claim 2, wherein, the virtual user object is a game character object; the similar virtual attribute object is the health bar object corresponding to the game character object; the matching image is an image including the health bar object.

7. The method according to claim 2, wherein, the matching image region includes a first image matching region and a second image matching region; Determining the first object image region according to the matching image region, including: Obtaining a first matching position of the first image matching region in the image, and obtaining a second matching position of the second image matching region in the image; When the position distance between the first matching position and the second matching position is less than or equal to a distance threshold, determining the first matching position or the second matching position as the target position, and determining the first object image region according to the target position; When the position distance between the first matching position and the second matching position is greater than the distance threshold, determining both the first matching position and the second matching position as the target position, and determining the first object image region according to the target position.

8. The method according to claim 7, wherein, Determining the first object image region according to the target position, including: When there are at least two target positions, obtaining a position image region corresponding to each target position in the image; the position image region is the region where the virtual user object is located; Merging the position image regions corresponding to each target position to obtain the first object image region.

9. The method according to claim 7, wherein, Determining the first object image region according to the target position, including: Obtaining the object width of the similar virtual attribute object and the object length of the similar virtual attribute object; Determining a second object image region of the similar virtual attribute object in the image according to the target position, the object width of the similar virtual attribute object, and the object length of the similar virtual attribute object; Determining the first object image region according to the second object image region and the virtual user object.

10. The method according to claim 9, wherein, Determining the first object image region according to the second object image region and the virtual user object, including: Obtaining the object height of the virtual user object, and obtaining the object length of the virtual user object; Determine the first object image area based on the object height of the virtual user object, the object length of the virtual user object, the regional position of the second object image area in the image, and the relative position between the virtual user object and the second object image area in the image.

11. The method according to claim 1, wherein, the hiding and displaying of the multimedia data in the first object image area includes: setting the first object image area as a mask image area; based on the mask image area, hiding and displaying the multimedia data in the first object image area.

12. An image data processing device, wherein, it includes: an identification module, configured to obtain a first object image area where a virtual user object is located in an image; the first object image area is obtained based on the detection of similar virtual attribute objects corresponding to a virtual user object cluster in the image, the virtual user object cluster includes the virtual user object, the first object image area is determined according to the positions of the detected similar virtual attribute objects in the image and the relative position relationship between the detected similar virtual attribute objects and the virtual user object, and the similar virtual attribute object is a health bar object corresponding to the virtual user object cluster; an acquisition module, configured to acquire multimedia data for independent display on the image; a display module, configured to hide and display the multimedia data in the first object image area, and superimpose and display the multimedia data in the background image area with the background image area; the background image area is the image area in the image except the first object image area.

13. The device according to claim 12, wherein, the identification module includes: a matching acquisition unit, configured to acquire a matching image including the similar virtual attribute object; a traversal unit, configured to slide and traverse the matching image in the image to obtain a matching image area corresponding to the matching image in the image; the matching image area has image similarity with the matching image; a region determination unit, configured to determine the first object image area according to the matching image area.

14. A computer device, including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 1-11.

15. A computer-readable storage medium, wherein, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1-11 is executed.

Images