A method, device, medium and equipment for displaying game auxiliary information

Abstract

The present invention provides a method, device, medium, and equipment for displaying game-assisted information, relating to the field of computer technology. The present invention determines a target reference image by matching the current user interface with a reference image in a game-assisted video. The target reference image is then used to determine game-assisted information to provide user operation prompts. This eliminates the need for users to repeatedly switch between the game application interface and the game strategy interface during gameplay, simplifying the use of game strategies and improving the convenience of user game operations, thereby enhancing the user's gaming experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, medium and equipment for displaying game auxiliary information. Background Art

[0002] There are currently many game strategy videos on video software. Players can refer to the game strategy videos to perform game operations, thereby improving their winning rate in the game.

[0003] Game guide videos are generally contained in video software. When players refer to game guide videos to perform game operations, they need to switch between the video software and the game software, or split the terminal device into two screens so that the terminal device can display the video software and the game software at the same time. However, whether frequently switching software or referring to game guide videos to perform game operations in a split-screen mode will affect the player's gaming experience.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for displaying game auxiliary information, a device for displaying game auxiliary information, a computer-readable storage medium and a device to address the problem in the prior art that players have poor gaming experience when performing game operations with reference to game auxiliary videos.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] According to a first aspect of an embodiment of the present invention, a method for displaying game auxiliary information is provided, wherein a user interface including a virtual game scene and virtual objects is provided through a terminal device, and the method includes:

[0008] Get a screenshot of the current user interface;

[0009] Matching the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image;

[0010] determining behavior data of the virtual object within a preset time period according to a current time point of the target reference image in the game auxiliary video, and generating corresponding game auxiliary information according to the behavior data;

[0011] The game auxiliary information is displayed in the current user interface to provide operation prompts to the user.

[0012] In an exemplary embodiment of the present invention, the behavior data includes a displacement vector of the virtual object, and the game auxiliary information includes a movement route of the virtual object;

[0013] The determining, based on the current time point of the target reference image in the game auxiliary video, the behavior data of the virtual object within a preset time period, and generating corresponding game auxiliary information based on the behavior data, includes:

[0014] Acquire, according to the current time point, multiple frames of intermediate reference images in the game auxiliary video corresponding to the preset time period;

[0015] extracting feature points of the virtual object in each of the intermediate reference images, and determining position information of the feature points;

[0016] Calculating an optical flow field between two adjacent intermediate reference images, and determining a displacement vector of the virtual object between the two adjacent intermediate reference images based on the position information of the feature points and the optical flow field;

[0017] A moving route of the virtual object is generated according to the displacement vector.

[0018] In an exemplary embodiment of the present invention, generating the movement route of the virtual object according to the displacement vector includes:

[0019] A moving route of the virtual object is obtained by smoothing a plurality of continuous displacement vectors.

[0020] In an exemplary embodiment of the present invention, matching the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image includes:

[0021] Extracting a plurality of first feature points from the virtual game scene area in the screenshot image, and extracting a plurality of second feature points from the virtual game scene area in each of the reference images;

[0022] Describing the local area around each of the first feature points and the local area around each of the second feature points respectively to obtain corresponding multiple first descriptors and multiple second descriptors;

[0023] Calculating the similarity between each of the first descriptors and each of the second descriptors corresponding to the local area, and using the reference image whose similarity satisfies a preset similarity condition as the target reference image. In an exemplary embodiment of the present invention, displaying the game auxiliary information in the current user interface includes:

[0024] A display window is provided in the current user interface, and the game auxiliary information is displayed in the display window.

[0025] In an exemplary embodiment of the present invention, the game auxiliary information further includes an estimated travel time of the movement route of the virtual object;

[0026] When displaying the game auxiliary information in the current user interface, the method further includes:

[0027] Calculating the estimated travel time based on the moving route and moving speed of the virtual object in the current user interface;

[0028] Displaying the estimated travel time of the moving route of the virtual object in the current user interface;

[0029] The estimated travel time of the moving route of the virtual object is updated in real time according to the movement of the virtual object.

[0030] In an exemplary embodiment of the present invention, before matching the screen capture image with a plurality of reference images in the game auxiliary video, the method further includes:

[0031] Decomposing the game auxiliary video into multiple frames of original images;

[0032] Calculating a grayscale histogram of each of the original images, and determining a cumulative histogram according to the frequency of each grayscale level in the grayscale histogram;

[0033] A mapping function is determined according to the cumulative histogram, and the grayscale of each pixel in each of the original images is changed according to the mapping function to obtain the corresponding reference images.

[0034] According to a second aspect of an embodiment of the present invention, a device for displaying game auxiliary information is provided, wherein a user interface including a virtual game scene and virtual objects is provided through a terminal device, and the device includes:

[0035] An image acquisition module is used to acquire a screenshot image of the current user interface;

[0036] An image matching module is used to match the screen capture image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the screen capture image;

[0037] an auxiliary information generation module, configured to determine the behavior data of the virtual object within a preset time period based on the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information based on the behavior data;

[0038] The auxiliary information display module is used to display the game auxiliary information in the current user interface to provide operation prompts to the user.

[0039] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processing unit, implements any of the above methods for displaying game auxiliary information.

[0040] According to a fourth aspect of the present invention, an electronic device is provided, comprising a processing unit; and a storage unit for storing executable instructions of the processing unit; wherein the processing unit is configured to execute any of the above-described methods for displaying game auxiliary information by executing the executable instructions.

[0041] The present invention determines the target reference image by matching the current user interface with the reference image in the game auxiliary video, and determines the game auxiliary information through the target reference image to provide operation prompts to the user. The user does not need to repeatedly switch between the game application interface and the game strategy interface during the game, which simplifies the way of using the game strategy, improves the convenience of the user in performing game operations, and thus enhances the user's gaming experience.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are incorporated herein and constitute a part thereof, illustrate embodiments consistent with the present invention, and together with the present invention, serve to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and it is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0044] Figure 1 A schematic diagram of a system architecture is shown in which a method for displaying game auxiliary information according to an embodiment of the present invention can be applied.

[0045] Figure 2 A flowchart of a method for displaying game auxiliary information in an embodiment of the present invention is shown.

[0046] Figure 3 A schematic diagram of a display interface for game auxiliary information designed using a method according to an embodiment of the present invention is shown.

[0047] Figure 4 A schematic diagram of a device for displaying game auxiliary information in an embodiment of the present invention is shown.

[0048] Figure 5A schematic structural diagram of an electronic device suitable for implementing an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present invention.

[0050] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The terms used in this invention are for the purpose of describing specific embodiments only and are not intended to limit the invention. The singular forms "a," "the," and "the" used in this invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0052] It should be understood that although the terms "first," "second," "third," etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0053] Next, the embodiments of the present invention are described in detail.

[0054] Figure 1 A schematic diagram of a system architecture is shown in which a method for displaying game auxiliary information according to an embodiment of the present invention can be applied.

[0055] like Figure 1 As shown, the system architecture 100 may include one or more terminal devices such as a desktop computer 101, a portable computer 102, a smart phone 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc. The terminal device may be any electronic device with a data processing function, which has a display screen for displaying game auxiliary videos and game operation interfaces to users. The electronic device includes but is not limited to the above-mentioned desktop computers, portable computers, smart phones and tablet computers, etc. The server 105 can communicate with the game software and video software to obtain relevant data of the current user interface and game auxiliary videos. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as needed. For example, the server 105 may be a server cluster consisting of multiple servers.

[0056] The method for displaying game auxiliary information provided in the embodiments of the present invention is generally executed by a terminal device, and accordingly, the display device for displaying game auxiliary information is generally located in the terminal device. However, those skilled in the art will readily appreciate that the method for displaying game auxiliary information provided in the embodiments of the present invention may also be executed by server 105, and accordingly, the display device for displaying game auxiliary information may also be located in server 105, and this exemplary embodiment does not specifically limit this.

[0057] This exemplary embodiment provides a method for displaying game auxiliary information. Figure 2 As shown, the method for displaying game auxiliary information may include the following steps S210 to S240:

[0058] Step S210. Get a screenshot of the current user interface;

[0059] Step S220. Matching the captured image with multiple reference frames in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured image;

[0060] Step S230. Determine the behavior data of the virtual object within a preset time period based on the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information based on the behavior data;

[0061] Step S240: Display the game auxiliary information in the current user interface to provide operation prompts to the user.

[0062] The present invention determines the target reference image by matching the current user interface with the reference image in the game auxiliary video, and determines the game auxiliary information through the target reference image to provide operation prompts to the user. The user does not need to repeatedly switch between the game application interface and the game strategy interface during the game, which simplifies the way of using the game strategy, improves the convenience of the user in performing game operations, and thus enhances the user's gaming experience.

[0063] The above steps of this exemplary embodiment are described in more detail below.

[0064] In step S210 , a screenshot image of the current user interface is obtained.

[0065] In an example embodiment of the present invention, the user interface may be the interface used by a user to operate a game via a terminal device, and the screenshot image may be an image obtained by capturing a screenshot of the terminal device interface. Before displaying game auxiliary information in the current user interface to provide user operation prompts, it is necessary to first determine the game status information using the screenshot image of the current user interface. The game status information may include status information of the virtual game scene and virtual objects in the current user interface.

[0066] In step S220, the screenshot image is matched with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the screenshot image.

[0067] In an exemplary embodiment of the present invention, the auxiliary game video may be a game guide video in video software. After acquiring the auxiliary game video, the auxiliary game video may be decomposed into multiple frames of original images. The grayscale histogram of each original image is then calculated. A cumulative histogram is determined based on the frequency of each grayscale level in the grayscale histogram. A mapping function is determined based on the cumulative histogram. The grayscale level of each pixel in each original image is modified according to the mapping function to obtain corresponding reference images.

[0068] Specifically, the user can first download and save the game auxiliary video to the terminal device, and then read the game auxiliary video frame by frame through the function provided by the video processing library to decompose the game auxiliary video into multiple frames of original images.

[0069] For example, after obtaining each original image, we can first count the number of pixels at each grayscale level in the original image, draw the grayscale histogram of the original image, then group each grayscale level in the grayscale histogram, calculate the frequency of each grayscale level in each group, and accumulate the frequency of each grayscale level in each group to obtain the cumulative frequency. Based on the grayscale levels in each group and the cumulative frequency of each grayscale level, we can draw a cumulative histogram. The cumulative histogram is normalized so that its value is between 0 and 255. Then, the mapping function is determined based on the normalized cumulative histogram. The mapping function can be determined by the following formula:

[0070]

[0071] in, represents the new gray level, Indicates the current gray level, represents the minimum gray level, Indicates the total number of pixels, Indicates the number of gray levels, Represents the cumulative pixel ratio of each gray level in the original image, Indicates rounding the calculation result in the following brackets.

[0072] After the mapping function is determined, each grayscale of each pixel in each original image can be mapped to a new grayscale, thereby obtaining corresponding reference images.

[0073] In addition to using histograms to enhance the contrast of each original image, the image quality of the original image can also be improved through other image enhancement algorithms. For example, the image quality can be improved through grayscale stretching, bilateral filtering, and Fourier transform. The present invention does not specifically limit this. Specifically, the contrast of each original image can be enhanced through grayscale stretching; then, each original image can be smoothed through bilateral filtering while retaining edge information and reducing the impact of noise; finally, each original image can be converted from the spatial domain to the frequency domain through Fourier transform, and each original image can be filtered in the frequency domain. Then, each original image can be converted back to the spatial domain to achieve image enhancement. Enhancing the contrast of each original image to obtain each reference image can improve the accuracy in the subsequent matching process of the captured image and the target reference image, thereby more accurately generating the corresponding game auxiliary information.

[0074] After obtaining each reference image, multiple first feature points can be extracted from the virtual game scene area in the screenshot image, and multiple second feature points can be extracted from the virtual game scene area in each reference image. The local area around each first feature point and the local area around each second feature point are described respectively to obtain corresponding multiple first descriptors and multiple second descriptors, and the similarity between each first descriptor and each second descriptor corresponding to the local area is calculated, and the reference image whose similarity meets the preset similarity condition is used as the target reference image.

[0075] Specifically, feature point detection algorithms such as SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), or SIFT (Scale Invariant Feature Transform) can be used to extract multiple first feature points of the virtual game scene area in the captured image and multiple second feature points of the virtual game scene area in each reference image. A descriptor algorithm is then used to numerically describe the local area surrounding each first feature point, generating multiple representative first descriptors. The local area surrounding each second feature point is also numerically described to generate multiple corresponding second descriptors. For example, a 128-dimensional first descriptor can be generated by calculating the directional gradient histogram around the first feature point, or a binary first descriptor can be generated by comparing the grayscale values ​​near the first feature point. Feature points can be robust and unique key points located at corners, edges, or texture-rich areas in the image. Descriptors can be represented in vector or matrix form.

[0076] Next, the similarity between each first descriptor and each second descriptor can be calculated using cosine similarity or other similarity measurement methods, and the reference image whose similarity to the captured image meets the preset matching conditions is used as the target reference image. Alternatively, the distance between each first descriptor and each second descriptor can be calculated using a distance metric such as Euclidean distance, Hamming distance, or angular distance, and the reference image corresponding to the second descriptor whose distance to the first descriptor meets the preset matching conditions is used as the target reference image. The preset matching conditions can be pre-set similarity values ​​or distance values ​​based on the scene requirements.

[0077] Optionally, the captured image and each reference image may be matched using a feature matching method such as a nearest neighbor matching algorithm, a best candidate matching algorithm, or RANSAC (Random Sample Consensus) to determine a target reference image.

[0078] In addition, the pixel matching values ​​between the screenshot image and each reference image can also be calculated through a deep learning model. First, the features of the screenshot image and each reference image can be extracted through a pre-trained convolutional neural network model, and then the matching values ​​between the screenshot image and each reference image can be calculated through methods such as cosine similarity or Euclidean distance. When the matching value with the screenshot image reaches a preset value, the corresponding reference image can be used as the target reference image, or the matching values ​​with the screenshot image can be sorted according to size, and the reference image corresponding to the matching value with the higher sorting can be selected as the target reference image.

[0079] Optionally, pixel-level matching values ​​between the screen capture image and each reference image can be determined by pixel-level comparison. Specifically, the screen capture image and each reference image can be scaled to the same resolution and color mode. A pixel-by-pixel comparison can then be performed between the screen capture image and each reference image. The matching value can then be determined by calculating the pixel-to-pixel ratio or average difference between the screen capture image and each reference image.

[0080] In this example, each reference image is obtained by enhancing the contrast and detail information of each original image, thereby improving the matching accuracy between the screenshot image and the target reference image. The target reference image is determined by calculating the matching value, thereby improving the comparison precision between images, thereby more accurately determining the target reference image and making the generated game auxiliary information more accurate.

[0081] In step S230, the behavior data of the virtual object within a preset time period is determined according to the current time point of the target reference image in the game auxiliary video, and corresponding game auxiliary information is generated according to the behavior data.

[0082] In an example embodiment of the present invention, the virtual object in the game assistance video may be a game character in the game scene shown in the game walkthrough video; the behavior data of the virtual object may include the displacement vector of the virtual object, and the game assistance information may include the movement route of the virtual object.

[0083] For example, you can first determine the current time point corresponding to the target reference image in the game auxiliary video, and then obtain multiple frames of intermediate reference images corresponding to the preset time period in the game auxiliary video based on the current time point; extract the feature points of the virtual object in each intermediate reference image, and determine the position information of the feature points; calculate the optical flow field between the intermediate reference images of two adjacent frames, and determine the displacement vector of the virtual object between the intermediate reference images of two adjacent frames based on the position information of the feature points and the optical flow field; and generate the movement route of the virtual object based on each displacement vector.

[0084] Specifically, when determining the current time point corresponding to the target reference image in the game auxiliary video, you can first determine the number of frames per second played by the game auxiliary video and calculate the frame rate of the game auxiliary video; obtain the position information corresponding to the target reference image in the game auxiliary video, where the position information of the target reference image can be represented by a frame number; then, determine the current time point corresponding to the target reference image in the game auxiliary video based on the frame number of the target reference image and the frame rate of the game auxiliary video. You can also determine the current time point based on the time point corresponding to the playback progress below the game auxiliary video.

[0085] After determining the current time point corresponding to the target reference image, the preset time period can be determined through the behavior data of the virtual game scene and the virtual object in the game auxiliary video. For example, the time it takes for the virtual object to complete a certain action in the virtual game scene can be used as the preset time period, or a fixed duration can be set as the preset time period according to actual needs. After obtaining multiple frames of intermediate reference images corresponding to the preset time period in the game auxiliary video, the virtual object can be determined through target detection algorithms, image segmentation algorithms or other algorithms for identifying virtual objects, and feature points of the virtual object can be extracted through feature extraction algorithms such as SIFT, SURF or FAST (Features from accelerated segment test), and then the position information of the feature points of the virtual object can be obtained, where the position information can be represented by coordinates.

[0086] After obtaining the position information of the virtual object's feature points, the optical flow field between two adjacent reference image frames can be calculated using feature extraction and optical flow calculation methods. The virtual object's displacement vector between adjacent frames is then determined based on the position information of the virtual object's feature points and the optical flow field. Finally, multiple consecutive displacement vectors of the virtual object within a preset time period are smoothed using a smoothing method such as Gaussian smoothing to determine the virtual object's movement path. This virtual object's movement path can be displayed as a curve.

[0087] In an exemplary embodiment of the present invention, a screenshot of the current user interface may be re-acquired after a preset time period, so as to obtain a new movement route by comparing the new screenshot with the game auxiliary video.

[0088] In this example, the movement route of the virtual object is generated according to the displacement vector of each adjacent frame of the virtual object in the game auxiliary video, so that the user can move the virtual object in the current user interface with reference to the movement route of the virtual object in the game auxiliary video, thereby improving the game winning rate and enhancing the user's gaming experience.

[0089] In step S240, the game auxiliary information is displayed in the current user interface to provide operation prompts to the user.

[0090] In one example embodiment of the present invention, a display window may be provided within the current user interface, and game auxiliary information may be displayed within the display window. The display window may be a full-screen window that occupies the entire current user interface and is highly transparent, or a small, semi-transparent window that is smaller than the current user interface. When displaying the movement route of a virtual object within the display window, the starting and ending points of the movement route may be indicated by arrows or color differences. When there are multiple virtual objects and corresponding movement routes, text or avatars may be added above the movement routes to help the user identify the virtual object corresponding to the movement route.

[0091] Optionally, an estimated travel time may be calculated based on the movement route and speed of the virtual object in the current user interface, and the estimated travel time for the virtual object's movement route may be displayed in the current user interface. The estimated travel time for the virtual object's movement route may also be updated in real time based on the movement of the virtual object. Other prompt information may also be displayed in the current user interface based on actual needs, and this disclosure is not specifically limited thereto.

[0092] Optionally, you can also add pause, resume, and close icons to the corners of the display window. When the user does not need to operate through the game auxiliary information, you can click the pause icon to pause the display of the game auxiliary information, and the display window will be temporarily minimized, so that only the continue and close icons are visible on the user interface. The user can click the continue icon to continue displaying the display window, or click the close icon to completely close the display window. The setting of the pause, open, and close icons can prevent the game auxiliary information from interfering with the user's operations when the user does not need to use the game auxiliary information, further enhancing the user's gaming experience.

[0093] like Figure 3 As shown, Figure 3 The following is a schematic diagram of a display interface for game auxiliary information designed using the method of an embodiment of the present invention. The display interface for game auxiliary information includes a virtual object's movement route 310, an estimated travel time display module 320, an operation icon module 330, and a virtual object's real-time location 340. The operation icon module 330 may include a pause icon 331, a resume icon 332, and a close icon 333. The virtual object's movement route 310 may be displayed in the form of a curve, the estimated travel time display module 320 may also be used to display other game auxiliary information, and the operation icon module 330 may also add or remove operation icons according to different needs to achieve different operation requirements.

[0094] In this example, game auxiliary information is displayed in the current user interface, so that users can directly refer to the game auxiliary information to play the game without opening the game auxiliary video, which reduces the operation steps when users refer to the game auxiliary video to play the game and improves the user's gaming experience.

[0095] In this exemplary embodiment, a device for displaying game auxiliary information is also provided. Figure 4 As shown, the game auxiliary information display device 400 may include an image acquisition module 410, an image matching module 420, an auxiliary information generation module 430 and an auxiliary information display module 440, wherein:

[0096] The image acquisition module 410 is used to acquire a screenshot image of the current user interface;

[0097] An image matching module 420 is configured to match the captured image with multiple reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured image.

[0098] An auxiliary information generation module 430 is configured to determine the behavior data of the virtual object within a preset time period based on the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information based on the behavior data;

[0099] The auxiliary information display module 440 is used to display the game auxiliary information in the current user interface to provide operation prompts to the user.

[0100] In an optional embodiment, the behavior data includes a displacement vector of the virtual object, and the game auxiliary information includes a movement route of the virtual object; the auxiliary information generation module 430 includes:

[0101] A reference image acquisition module, configured to acquire, according to the current time point, a plurality of intermediate reference images in the game auxiliary video corresponding to the preset time period;

[0102] a position information determination module, configured to extract feature points of the virtual object in each of the intermediate reference images and determine position information of the feature points;

[0103] a displacement vector determination module, configured to calculate an optical flow field between two adjacent intermediate reference images, and determine a displacement vector of the virtual object between the two adjacent intermediate reference images based on the position information of the feature points and the optical flow field;

[0104] A movement route generating module is used to generate a movement route of the virtual object according to the displacement vector.

[0105] In an optional embodiment, the movement route generation module includes:

[0106] The displacement vector processing module is used to perform smoothing processing on a plurality of continuous displacement vectors to obtain a moving route of the virtual object.

[0107] In an optional embodiment, the image matching module 420 includes:

[0108] a feature point extraction module, configured to extract a plurality of first feature points from the virtual game scene region in the captured image, and to extract a plurality of second feature points from the virtual game scene region in each of the reference images;

[0109] a descriptor generation module, configured to describe the local area around each of the first feature points and the local area around each of the second feature points, respectively, to obtain corresponding multiple first descriptors and multiple second descriptors;

[0110] The target image determination module is used to calculate the similarity between each of the first descriptors and each of the second descriptors corresponding to the local area, and use the reference image whose similarity meets the preset similarity condition as the target reference image.

[0111] In an optional implementation, the auxiliary information display module 440 includes:

[0112] A display window display module is used to provide a display window in the current user interface and display the game auxiliary information in the display window.

[0113] In an optional embodiment, the game auxiliary information further includes an estimated travel time of the virtual object's moving route; the auxiliary information display module 440 further includes:

[0114] a travel time determination module, configured to calculate the estimated travel time based on the moving route and moving speed of the virtual object in the current user interface;

[0115] A travel time display module, configured to display the estimated travel time of the moving route of the virtual object in the current user interface;

[0116] The travel time updating module is used to update the estimated travel time of the moving route of the virtual object in real time according to the movement of the virtual object.

[0117] In an optional embodiment, the image matching module 420 further includes:

[0118] A video decomposition module, configured to decompose the game auxiliary video into multiple frames of original images;

[0119] a cumulative histogram determination module, configured to calculate a grayscale histogram of each of the original images, and determine a cumulative histogram based on the frequency of each grayscale level in the grayscale histogram;

[0120] The reference image generation module is used to determine a mapping function according to the cumulative histogram, and change the grayscale of each pixel in each original image according to the mapping function to obtain the corresponding reference image.

[0121] The specific details of each module in the above-mentioned game auxiliary information display device have been described in detail in the corresponding game auxiliary information display method, so they will not be repeated here.

[0122] The exemplary embodiments of the present invention further provide a computer-readable storage medium having stored thereon a program product capable of implementing the methods described above in this specification. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product comprising program code that, when executed on an electronic device, causes the electronic device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above. This program product may be implemented in the form of a portable compact disc read-only memory (CD-ROM) and include the program code, and may be executed on an electronic device, such as a personal computer. However, the program product of the present invention is not limited thereto; in this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0123] The program product may utilize any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0124] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0126] Program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C#, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0127] The exemplary embodiment of the present invention also provides an electronic device capable of implementing the above method. Figure 5 An electronic device 400 according to this exemplary embodiment of the present invention will be described. Figure 5 The electronic device 500 shown is merely an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0128] like Figure 5 As shown, electronic device 500 may be implemented as a general-purpose computing device. Components of electronic device 500 may include, but are not limited to, at least one processing unit 510, at least one storage unit 520, a bus 530 connecting various system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0129] The storage unit 520 stores program codes that can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section above. For example, the processing unit 510 may execute Figure 2 The storage unit 520 may be used to store the screenshot image of the current user interface, the game auxiliary video, and the ultimately obtained game auxiliary information, etc., which is not limited in the present invention.

[0130] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 521 and / or a cache memory unit (Cache) 522 , and may further include a read-only memory unit (ROM) 523 .

[0131] The storage unit 520 may also include a program / utility 524 having a set (at least one) of program modules 525, such program modules 525 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0132] Bus 530 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0133] The electronic device 500 can also communicate with one or more external devices 600 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 500, and / or any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can occur via an input / output (I / O) interface 550. Furthermore, the electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 via a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems. From the above description of the embodiments, it will be readily apparent to those skilled in the art that the example embodiments described herein can be implemented via software or software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiment of the present invention.

[0134] Furthermore, the figures above are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0135] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0136] Other embodiments of the invention will readily occur to those skilled in the art upon consideration and practice of the invention disclosed herein. The invention is intended to cover any variations, uses, or adaptations of the invention that follow from the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The invention and embodiments are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0137] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for displaying game auxiliary information, characterized in that: Providing a user interface including a virtual game scene and virtual objects through a terminal device, the method comprising: Get a screenshot of the current user interface; Matching the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image; determining behavior data of the virtual object within a preset time period according to a current time point of the target reference image in the game auxiliary video, and generating corresponding game auxiliary information according to the behavior data; Providing a display window in the current user interface, and displaying the game auxiliary information in the display window to provide operation prompts to the user; The display window is provided with a pause icon, a resume icon, and a close icon for controlling the display window; the pause icon is used to minimize the display window after being triggered by the user, and the resume icon and the close icon are retained on the current user interface in the minimized state; the resume icon is used to continue displaying the display window after being triggered by the user, and the close icon is used to terminate the display of the game auxiliary information; Before matching the captured screen image with a plurality of reference images in the game auxiliary video, the method further includes: Decomposing the game auxiliary video into multiple frames of original images; Calculating a grayscale histogram of each of the original images, and determining a cumulative histogram according to the frequency of each grayscale level in the grayscale histogram; determining a mapping function according to the cumulative histogram, and changing the grayscale of each pixel in each of the original images according to the mapping function to obtain the corresponding reference images; The behavior data includes a displacement vector of the virtual object, and the game auxiliary information includes a movement route of the virtual object; The determining, based on the current time point of the target reference image in the game auxiliary video, the behavior data of the virtual object within a preset time period, and generating corresponding game auxiliary information based on the behavior data, includes: Acquire, according to the current time point, multiple frames of intermediate reference images in the game auxiliary video corresponding to the preset time period; extracting feature points of the virtual object in each of the intermediate reference images, and determining position information of the feature points; Calculating an optical flow field between two adjacent intermediate reference images, and determining a displacement vector of the virtual object between the two adjacent intermediate reference images based on the position information of the feature points and the optical flow field; A moving route of the virtual object is generated according to the displacement vector.

2. The method for displaying game auxiliary information according to claim 1, wherein: Generating the movement route of the virtual object according to the displacement vector includes: A moving route of the virtual object is obtained by smoothing a plurality of continuous displacement vectors.

3. The method for displaying game auxiliary information according to claim 1, wherein: The matching of the captured screen image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the captured screen image includes: Extracting a plurality of first feature points from the virtual game scene area in the screenshot image, and extracting a plurality of second feature points from the virtual game scene area in each of the reference images; Describing the local area around each of the first feature points and the local area around each of the second feature points respectively to obtain corresponding multiple first descriptors and multiple second descriptors; The similarity between each of the first descriptors and each of the second descriptors corresponding to the local area is calculated, and a reference image whose similarity satisfies a preset similarity condition is used as the target reference image.

4. The method for displaying game auxiliary information according to claim 1, wherein: The game auxiliary information also includes an estimated travel time of the movement route of the virtual object; When displaying the game auxiliary information in the current user interface, the method further includes: Calculating the estimated travel time based on the moving route and moving speed of the virtual object in the current user interface; Displaying the estimated travel time of the moving route of the virtual object in the current user interface; The estimated travel time of the moving route of the virtual object is updated in real time according to the movement of the virtual object.

5. A device for displaying game auxiliary information, characterized in that: The method for displaying game auxiliary information according to any one of claims 1 to 4 is applied to provide a user interface including a virtual game scene and virtual objects through a terminal device, the device comprising: An image acquisition module is used to acquire a screenshot image of the current user interface; An image matching module is used to match the screen capture image with multiple frames of reference images in the game auxiliary video to determine a target reference image that matches the virtual game scene in the screen capture image; an auxiliary information generation module, configured to determine the behavior data of the virtual object within a preset time period based on the current time point of the target reference image in the game auxiliary video, and generate corresponding game auxiliary information based on the behavior data; The auxiliary information display module is used to display the game auxiliary information in the current user interface to provide operation prompts to the user.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing unit, the method for displaying game auxiliary information according to any one of claims 1 to 4 is implemented.

7. An electronic device, characterized in that: include: processing unit; as well as a storage unit, configured to store executable instructions of the processing unit; Wherein, the processing unit is configured to execute the method for displaying game auxiliary information according to any one of claims 1 to 4 by executing the executable instructions.

Images