Object detection, cloud game control method, device, medium, equipment and product

Abstract

The application discloses an object detection, cloud game control method, device, medium, equipment and product, relates to the technical field of cloud games, and can be applied to the technical field of blockchains, map Internet of Vehicles and the like. The object detection method comprises the following steps: acquiring a to-be-detected image and template feature data; detecting at least one feature point in the to-be-detected image, and generating index information of each feature point; performing feature modeling on the at least one feature point, and generating point set feature data in a target format; performing at least one matching process on the template feature data and the point set feature data, so as to successively determine a single group of feature points; after a single group of feature points is matched each time, hiding and processing sub-feature data corresponding to the single group of feature points in the point set feature data according to the index information and the target format; and determining an object detection result according to the single group of feature points. The application can effectively reduce the calculation pressure and time consumption when an object is detected based on an image, can accelerate the progress of cloud games, and can improve the experience of cloud games.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an object detection, cloud gaming control method, apparatus, medium, device, and product. Background Technology

[0002] Object detection tasks typically involve identifying objects from images. For example, in cloud gaming, there's the task of detecting virtual characters or controls from images of the cloud gaming interface. Currently, some related technologies employ lazy matching for object detection.

[0003] In the relevant scheme, firstly, a batch of feature points are detected from the image, and then object matching is performed after modeling. Then, the corresponding areas of the matched feature points in the image are filled with solid colors, and the next batch of feature points is detected, modeled, and then object matching is performed. In this way, feature point detection, modeling, matching, and filling are performed in a loop to perform object detection.

[0004] Related technologies suffer from high computational pressure and long processing time when performing object detection based on images. In particular, for cloud gaming, this slows down the cloud gaming process and reduces the cloud gaming experience. Summary of the Invention

[0005] This application provides an object detection, cloud gaming control method and related apparatus, which can effectively reduce the computational pressure and time consumption when performing object detection based on images, accelerate the cloud gaming process, and improve the cloud gaming experience.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] According to one embodiment of this application, an object detection method is provided, comprising: acquiring template feature data of template feature points of a predetermined object in an image to be detected; detecting at least one feature point included in the image to be detected and generating index information corresponding to each feature point; performing feature modeling on the at least one feature point to generate point set feature data in a target format; performing at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points matching the template feature points from the feature points, wherein, after each matching of a single set of feature points, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; and determining the object detection result corresponding to the image to be detected based on the single set of feature points.

[0008] According to one embodiment of this application, an object detection device includes: an acquisition module for acquiring template feature data of template feature points of a predetermined object in an image to be detected; a detection module for detecting at least one feature point in the image to be detected and generating index information corresponding to each feature point; a modeling module for performing feature modeling on the at least one feature point to generate point set feature data in a target format; a matching module for performing at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points matching the template feature points from the feature points, wherein after each matching of a single set of feature points, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; and a determination module for determining the object detection result corresponding to the image to be detected based on the single set of feature points.

[0009] In some embodiments of this application, the matching module includes: a matching unit, configured to perform matching processing on the template feature data and the point set feature data to obtain a single set of feature points that match the template feature points among the at least one feature points, wherein the single set of feature points is a feature point corresponding to a single predetermined object; a positioning unit, configured to determine sub-feature data corresponding to the single set of feature points in the point set feature data according to the index information and the target format corresponding to the single set of feature points; a hiding unit, configured to perform hiding processing on the sub-feature data in the point set feature data to obtain hidden feature data; and a repeating unit, configured to replace the point set feature data with the hidden feature data and repeatedly execute the above matching processing to hiding processing steps to obtain at least one set of single feature points that match the template feature points.

[0010] In some embodiments of this application, the index information is a number, and the target format point set feature data is generated by concatenating at least one two-dimensional array of target size according to the number of feature points, with each image channel of the feature point corresponding to one two-dimensional array; the positioning unit is used to: multiply the number corresponding to each feature point in the single set of feature points, the target size, and the number of image channels to obtain the multiplication result corresponding to each feature point in the single set of feature points; and determine the sub-feature data corresponding to each feature point in the single set of feature points from the point set feature data based on the multiplication result corresponding to each feature point in the single set of feature points.

[0011] In some embodiments of this application, the hiding unit is used to: reset the sub-feature data in the point set feature data to predetermined data to obtain the hidden feature data, wherein the predetermined data includes either zero or infinity.

[0012] In some embodiments of this application, the modeling module includes: a neighborhood determination unit, configured to select a neighborhood corresponding to each feature point in the image to be detected for the at least one feature point; a partitioning unit, configured to divide the neighborhood corresponding to each feature point into a predetermined number of sub-domains, and perform feature extraction processing on the sub-domains respectively to obtain a predetermined number of sub-domain feature data corresponding to each feature point; and a generation unit, configured to generate point set feature data in the target format based on the predetermined number of sub-domain feature data corresponding to each feature point.

[0013] In some embodiments of this application, the target format point set feature data is generated based on at least one two-dimensional array of target size, and each two-dimensional array corresponds to one feature point; the generation unit is configured to: take a predetermined number of subdomain feature data corresponding to each feature point as array elements to generate a two-dimensional array of target size corresponding to each feature point; and generate the target format point set feature data based on the two-dimensional array of target size corresponding to each feature point.

[0014] According to one embodiment of this application, a cloud gaming control method includes: acquiring a game interface image of a target cloud game; using the game interface image as an image to be detected, performing object detection according to the object detection method described in any embodiment of this application to obtain an object detection result; and controlling the target cloud game based on the object detection result.

[0015] According to one embodiment of this application, a cloud gaming control device includes: a data acquisition unit for acquiring game interface images of a target cloud game; a matching unit for using the game interface images as images to be detected and performing object detection according to the object detection method described in any embodiment of this application to obtain object detection results; and a control unit for controlling the target cloud game based on the object detection results.

[0016] In some embodiments of this application, the cloud gaming control device further includes: a script acquisition unit, configured to: acquire a cloud gaming script file, wherein the cloud gaming script file is configured with predetermined detection logic and predetermined control logic; a matching unit, configured to: according to the predetermined detection logic, take the game interface image as the image to be detected, perform object detection according to the object detection method described in any embodiment of this application, and obtain an object detection result; and a control unit, configured to: according to the predetermined control logic, control the target cloud game based on the object detection result.

[0017] In some embodiments of this application, the control unit is configured to: generate a control instruction corresponding to the object detection result according to the predetermined control logic; and send the control instruction to the cloud gaming backend so that the cloud gaming backend executes the game operation corresponding to the control instruction.

[0018] In some embodiments of this application, the acquisition unit is configured to: if the target cloud game is detected to have entered a specific game scene, acquire the game interface image of the target cloud game in the specific game scene.

[0019] According to another embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a computer's processor, causes the computer to perform the methods described in the embodiments of this application.

[0020] According to another embodiment of this application, an electronic device includes: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the methods described in the embodiments of this application.

[0021] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0022] In this embodiment: template feature data of template feature points of a target image and a predetermined object are obtained; at least one feature point included in the target image is detected, and index information corresponding to each feature point is generated; feature modeling is performed on the at least one feature point to generate point set feature data in a target format; the template feature data and the point set feature data are matched at least once to determine a single set of feature points matching the template feature point from the feature points one by one, wherein, after each matching of a single set of feature points, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; and the object detection result corresponding to the target image is determined according to the single set of feature points.

[0023] In this way, by hiding the sub-feature data corresponding to the single set of feature points in the point set feature data according to the index information and target format after each matching process, and successively determining the single set of feature points matching the template feature points from the feature points, the object detection only needs to perform one feature point detection and one feature modeling. The object detection result can be determined based on the single set of feature points without the need for image filling operations. This effectively reduces the computational pressure and time consumption when performing image-based object detection. In particular, it can effectively accelerate the cloud gaming process and improve the cloud gaming experience in cloud gaming scenarios. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of a system to which embodiments of this application can be applied is shown.

[0026] Figure 2 A flowchart of an object detection method according to an embodiment of this application is shown.

[0027] Figure 3 A schematic diagram of image sampling according to an embodiment of this application is shown.

[0028] Figure 4 A schematic diagram of feature modeling according to an embodiment of this application is shown.

[0029] Figure 5 A schematic diagram of a hidden process according to an embodiment of this application is shown.

[0030] Figure 6 A flowchart of a cloud gaming control method according to an embodiment of this application is shown.

[0031] Figure 7 A schematic diagram of a cloud gaming architecture according to an embodiment of this application is shown.

[0032] Figure 8 A schematic diagram of a cloud gaming control architecture according to an embodiment of this application is shown.

[0033] Figure 9 A flowchart illustrating script parsing and game control according to an embodiment of this application is shown.

[0034] Figure 10A schematic diagram of object detection logic according to an embodiment of this application is shown.

[0035] Figure 11 A schematic diagram of object detection logic according to an embodiment of this application is shown.

[0036] Figure 12 The image shows a game interface in one scenario.

[0037] Figure 13 It shows Figure 12 Another game interface image in the scene.

[0038] Figure 14 It shows Figure 12 Another game interface image in the scene.

[0039] Figure 15 This shows an image of a game interface in another scenario.

[0040] Figure 16 It shows Figure 15 A game interface image representing the object detection process in the relevant technologies of the scene.

[0041] Figure 17 It shows Figure 15 Another game interface image in the context of object detection technology.

[0042] Figure 18 It shows Figure 15 The embodiments of this application are used to perform object detection on game interface images in a specific scenario.

[0043] Figure 19 A flowchart comparing the object detection process in a scenario using related technologies and embodiments of this application is shown.

[0044] Figure 20 A block diagram of an object detection apparatus according to an embodiment of this application is shown.

[0045] Figure 21 A block diagram of a cloud gaming control device according to another embodiment of this application is shown.

[0046] Figure 22 A block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] It is understood that in the specific implementation of this application, data related to content information and interactive behavior information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0049] Figure 1 A schematic diagram of a system 100 to which embodiments of this application can be applied is shown. For example... Figure 1 As shown, system 100 may include server 110 and terminal 120.

[0050] Server 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0051] Terminal 120 can be any device, including but not limited to mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, VR / AR devices, smartwatches, and computers, etc. In some embodiments, terminal 120 and server 110 can be nodes in a blockchain network.

[0052] In one embodiment of this example, server 110 or terminal 120 may: acquire template feature data of template feature points of a target image and a predetermined object; detect at least one feature point included in the target image and generate index information corresponding to each feature point; perform feature modeling on the at least one feature point to generate point set feature data in a target format; perform at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points matching the template feature points from the feature points, wherein, after each matching of a single set of feature points, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; and determine the object detection result corresponding to the target image based on the single set of feature points.

[0053] In some implementations, the image to be detected can be an image in a map-based vehicle networking platform. In some implementations, the image to be detected can be an image in a cloud gaming platform.

[0054] In one embodiment of this example, server 110 may: acquire game interface images of the target cloud game; use the game interface images as images to be detected, perform object detection according to the object detection method described in any embodiment of this application, and obtain object detection results; and control the target cloud game according to the object detection results.

[0055] Figure 2 A flowchart illustrating an embodiment of an object detection method according to this application is shown schematically. The object detection method can be performed by any device, such as... Figure 1 The server 110 or terminal 120 shown.

[0056] like Figure 2 As shown, the object detection method may include steps S210 to S250.

[0057] Step S210: Obtain template feature data of template feature points of the image to be detected and the predetermined object; Step S220: Detect at least one feature point included in the image to be detected and generate index information corresponding to each feature point; Step S230: Perform feature modeling on at least one feature point to generate point set feature data in the target format; Step S240: Perform at least one matching process between the template feature data and the point set feature data to determine a single set of feature points matching the template feature points from the feature points one by one. Each time a single set of feature points is matched, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; Step S250: Determine the object detection result corresponding to the image to be detected based on the single set of feature points.

[0058] The image to be detected is the image that needs to be detected, such as a cloud gaming interface image or a map image. The image to be detected can be obtained from a predetermined location or by taking a screenshot in real time.

[0059] A pre-defined object is an object in a pre-defined image. In one example, a pre-defined object could be a virtual character or control in a specified cloud gaming interface image.

[0060] Template feature points are the feature points that serve as a template. Template feature points can be feature points obtained by detecting the template object image corresponding to a single predetermined object. Template feature data can be generated by performing feature modeling on the template feature points.

[0061] By performing a single detection on the image to be detected, we can obtain feature points in the image, that is, obtain at least one feature point.

[0062] For each feature point, corresponding index information can be generated. Each feature point can be uniquely identified through the index information, which can include information such as a number.

[0063] Perform feature modeling on all at least one feature point to generate point set feature data in the target format. The feature data includes sub-feature data corresponding to each feature point. The target format can be specified according to the actual situation.

[0064] A single set of feature points is a group of feature points. For example, at least one feature point is 500 feature points, the template feature points include 50 feature points, and so on. A single set of feature points includes 50 feature points out of 500 feature points.

[0065] By performing at least one matching process between template feature data and point set feature data, single sets of feature points that match the template feature points can be determined successively from the feature points. Using the generated feature point index information and the specified target format, the sub-feature data corresponding to each feature point can be accurately located from the point set feature data. Furthermore, after each matching of a single set of feature points from at least one feature point, the sub-feature data corresponding to the matched single set of feature points in the point set feature data can be accurately hidden according to the index information and target format before proceeding to the next matching. This matching method allows object detection to perform only one feature point detection and one feature modeling operation.

[0066] In this way, based on steps S210 to S250, after each matching of a single set of feature points in at least one matching process, the sub-feature data corresponding to the matched single set of feature points in the point set feature data is hidden according to the index information and target format. The single set of feature points matching the template feature points is determined from the feature points one by one. This allows object detection to only require one feature point detection and one feature modeling. The object detection result can be determined based on a single set of feature points without the need for image filling operations. This effectively reduces the computational pressure and time consumption when performing object detection based on images. In particular, it can effectively accelerate the cloud gaming process and improve the cloud gaming experience in cloud gaming scenarios.

[0067] The following description Figure 2 Other embodiments corresponding to the specific processes of each step performed during object detection in the embodiments.

[0068] Step S210: Obtain template feature data of template feature points of the image to be detected and the predetermined object.

[0069] In one implementation, the image to be detected can be obtained from a predetermined location; in another implementation, the image to be detected is obtained by taking a real-time screenshot.

[0070] In one embodiment, template feature data of template feature points of a predetermined object is obtained from a predetermined location. In another embodiment, a template object image of the predetermined object can be obtained (for example, the predetermined object is a control, and the template object image is the template control image of the control). The template object image is detected to obtain the template feature points of the predetermined object, and the template feature points are modeled to obtain the template feature data.

[0071] Step S220: Detect at least one feature point in the image to be detected, and generate index information corresponding to each feature point.

[0072] Detection in an image involves identifying feature points from contours and corners within the image, resulting in at least one detected feature point. The detected feature points include those corresponding to a predetermined object within the image, as well as other noise-related feature points.

[0073] For example, the image to be detected may include at least one predetermined object, such as the predetermined object being... Figure 15 The virtual characters of the three tennis robots in the cloud gaming interface image shown are, for example... Figure 15 The cloud gaming interface image shown may also include other virtual characters, and at least one detected feature point may include feature points corresponding to the virtual characters of the three tennis robots as well as noise feature points corresponding to other virtual characters.

[0074] Furthermore, in one embodiment, see [reference] Figure 3 An image pyramid can be constructed by upsampling and downsampling the image to be detected using filters (e.g., ...). Figure 3 The method involves creating an image pyramid composed of images of varying sizes. A size-invariant convolutional kernel is used to extract feature points from the image pyramid, identifying at least one feature point within the image to be detected, thus improving the accuracy of feature point detection. The size-invariant convolutional kernel can be a pre-trained kernel specifically designed for feature point detection.

[0075] For example, the Scale-invariant Feature Transform (SIFT) algorithm or the Speeded Up Robust Features (SURF) algorithm can be used to construct an image pyramid by upsampling and downsampling the image to be detected through filters, and feature points can be extracted from the image pyramid using size-invariant convolution kernels to obtain at least one feature point included in the image to be detected.

[0076] Step S230: Perform feature modeling on at least one feature point to generate point set feature data in the target format.

[0077] In one embodiment, step S230 involves performing feature modeling on at least one feature point to generate point set feature data in a target format. This can be achieved by extracting texture feature data corresponding to each feature point, combining the texture feature data corresponding to each feature point, and generating point set feature data in the target format.

[0078] In one embodiment, step S230, which involves performing feature modeling on at least one feature point to generate point set feature data in a target format, includes: selecting a neighborhood corresponding to each feature point in the image to be detected for the at least one feature point; dividing the neighborhood corresponding to each feature point into a predetermined number of sub-domains, and performing feature extraction processing on each sub-domain to obtain a predetermined number of sub-domain feature data corresponding to each feature point; and generating point set feature data in a target format based on the predetermined number of sub-domain feature data corresponding to each feature point.

[0079] participate Figure 4 For each feature point, a neighborhood corresponding to each feature point can be selected in the image to be detected, centered on the feature point and according to a predetermined neighborhood range. Figure 4 The diagram illustrates a square neighborhood of a feature point. It is understood that the shape of the neighborhood can be specified as needed in other implementations. The neighborhood corresponding to each feature point is divided into a predetermined number of sub-regions, for example... Figure 4 The square neighborhood is divided into 16 sub-domains. Feature extraction is performed on each sub-domain to obtain a predetermined number of sub-domain feature data for each feature point. For example, for... Figure 4 Feature extraction was performed on each of the 16 subdomains to obtain feature data for each of the 16 subdomains corresponding to the feature points.

[0080] Based on a predetermined number of subdomain feature data corresponding to each feature point, point set feature data in the target format is generated, thereby realizing neighborhood feature modeling to obtain point set feature data. In this way, the accuracy of matching specific values ​​can be further improved.

[0081] In one embodiment, the target format point set feature data is generated based on at least one target size two-dimensional array, where each two-dimensional array corresponds to one feature point; generating the target format point set feature data based on a predetermined number of subdomain feature data corresponding to each feature point includes: using the predetermined number of subdomain feature data corresponding to each feature point as array elements to generate a target size two-dimensional array corresponding to each feature point; and generating the target format point set feature data based on the target size two-dimensional array corresponding to each feature point.

[0082] For example, see Figure 4The 16 subdomain feature data are used as array elements. If each subdomain feature data includes 4 data points, a 16*4=64-dimensional two-dimensional array is generated. This generates a 64-dimensional (i.e., target size) two-dimensional array corresponding to each feature point. By combining all the two-dimensional arrays of feature points according to the target format, the point set feature data in the target format can be obtained. In this way, by representing feature points with two-dimensional arrays, the accuracy of feature point matching is further improved, and the sub-feature data of each feature point can be accurately located in subsequent steps.

[0083] In one embodiment, see Figure 4 According to such Figure 4 The main direction of the feature point indicated by the arrow divides the square neighborhood of the feature point into 16 sub-regions evenly. For example, if the scale of the feature point in this application is S and the side length of the square neighborhood is 20S, the square neighborhood can be evenly divided into 16 square sub-regions with a side length of 5S.

[0084] Feature extraction is performed on each subdomain separately. Specifically, this may include: calculating Haar wavelet features (Haar wavelet characteristics) for 25 pixels in each subdomain in the horizontal direction (parallel to the main direction) and the vertical direction (perpendicular to the main direction). These Haar wavelet features include the sum of horizontal values ​​∑dy, the sum of horizontal absolute values ​​∑|dy|, the sum of vertical values ​​∑dx, and the sum of vertical absolute values ​​∑|dx|. Thus, the subdomain feature data for each subdomain includes 4 dimensions: ∑dy, ∑|dy|, ∑dx, and ∑|dx|. The 16 subdomains have a total of 16*4=64 dimensions.

[0085] Step S240: Perform at least one matching process between the template feature data and the point set feature data to determine the single set of feature points that match the template feature points from the feature points one by one. Each time a single set of feature points is matched, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format.

[0086] In one embodiment, step S240 involves performing at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points that match the template feature points from the feature points, including:

[0087] The template feature data is matched with the point set feature data to obtain at least one set of feature points that match the template feature points. Each set of feature points corresponds to a single predetermined object. Based on the index information and target format of the set of feature points, the sub-feature data corresponding to the set of feature points is determined in the point set feature data. The sub-feature data in the point set feature data is hidden to obtain hidden feature data. The hidden feature data replaces the point set feature data, and the above matching to hiding steps are repeated to obtain at least one set of feature points that match the template feature points.

[0088] Matching template feature data with point set feature data can be performed using similarity metrics such as Euclidean distance. This involves measuring the similarity between sub-feature data in the template feature data and sub-feature data in the point set feature data, and outputting the index information of single-group feature points corresponding to sub-feature data in the point set feature data whose similarity to sub-feature data in the template feature data is less than a predetermined threshold. This process then matches the first group of single-group feature points.

[0089] Subsequently, based on the index information and target format corresponding to a single set of feature points, the sub-feature data corresponding to that single set of feature points can be located in the point set feature data. This located sub-feature data is then hidden to obtain the hidden feature data. For example, see [link to relevant documentation]. Figure 5 This process hides all 16 subdomain feature data in the sub-feature data corresponding to a certain feature point.

[0090] By replacing the point set feature data in the first step of the matching process with the hidden feature data, the above steps of matching to hiding are repeated, thereby determining at least one set of single feature points. The number of times the above steps of matching to hiding are repeated can be specified in advance, or it can be left unspecified and the repetition can end when no single set of feature points is matched.

[0091] This method can precisely hide only the sub-feature data corresponding to a single set of feature points. After hiding, the feature data retains the complete sub-feature data of the remaining feature points. By cyclically matching single sets of feature points and locating their corresponding sub-feature data for hiding, the accuracy of the final matched single set of feature points can be improved.

[0092] In one example, the template feature data and the point set feature data can be matched according to the following code logic to obtain at least one set of feature points that match the template feature points.

[0093]

[0094] In this example, through this code logic, each time the template feature data is matched with the point set feature data, the similarity between the sub-feature data in the template feature data and the sub-feature data in the point set feature data is calculated using Euclidean distance (similarity). The index information (queryIdx) of a single set of feature points corresponding to the sub-feature data in the point set feature data whose similarity to the sub-feature data in the template feature data is less than a predetermined threshold is output. Then, a single set of matched feature points is output through the index information (queryIdx). As described in the above code logic, each matching process can output "the index information (queryIdx, i.e., the index information of a single set of feature points) of the feature points of the image to be detected corresponding to this match" and "the index information (trainIdx, i.e., the index information of the template feature points corresponding to the single set of feature points) of the template object image corresponding to this match".

[0095] In one embodiment, the index information is a number, and the target format point set feature data is generated by concatenating at least one two-dimensional array of target size according to the feature point numbers, with each image channel of the feature point corresponding to a two-dimensional array; based on the index information and target format corresponding to a single set of feature points, the sub-feature data corresponding to a single set of feature points is determined in the point set feature data, including: multiplying the number, target size, and number of image channels corresponding to each feature point in the single set of feature points to obtain the multiplication result corresponding to each feature point in the single set of feature points; and based on the multiplication result corresponding to each feature point in the single set of feature points, the sub-feature data corresponding to each feature point in the single set of feature points is determined from the point set feature data.

[0096] For example, if feature point A in a single set of feature points is numbered 20 and the target size is 64, and the image to be detected is a 3-channel image, then the number of image channels is 3, that is, each specific point corresponds to a two-dimensional array of 3 channels, and the result of multiplication is 20*64*3=3840.

[0097] Then, when determining the sub-feature data corresponding to feature point A based on the multiplication result, since the point set feature data is generated by concatenating at least one two-dimensional array of target size according to the feature point number, subtracting the product of target size 64 and image channel number 3 (192) from the multiplication result 3840 yields 3648. Thus, the array elements between 3648 and 3840 in the point set feature data can be determined as the sub-feature data corresponding to feature point A.

[0098] In one embodiment, the process of hiding sub-feature data in point set feature data to obtain hidden feature data includes: resetting the sub-feature data in point set feature data to predetermined data to obtain hidden feature data, wherein the predetermined data includes either zero or infinity.

[0099] participate Figure 5Each feature point corresponds to a 16*4=64-dimensional two-dimensional array as its sub-feature data. The point set feature data includes at least one two-dimensional array corresponding to at least one feature point. Each element in the two-dimensional array corresponding to each feature point in the matched single-set feature data is reset to either zero or infinity. Thus, the point set feature data is transformed into hidden feature data. In this way, by resetting the elements to either zero or infinity, when matching based on the hidden feature data and the template feature data, the similarity between the arrays in the hidden feature data that are reset to zero or infinity and the template feature data will be extremely small, thus preventing the previously matched single-set feature points from being matched again.

[0100] In one example, the code logic can be followed to locate the sub-feature data corresponding to a single set of feature points in the point set feature data based on the index information and target format of the single set of feature points, and then hide the located sub-feature data in the point set feature data.

[0101]

[0102] This code logic executes the formula: nOffset = szDMatchs[i].trainIdx * m_imgSourceCompute.cols * nChannels. This multiplies the index (szDMatchs[i].trainIdx), target size (m_imgSourceCompute.cols), and number of image channels (nChannels) of the i-th feature point in a single set of feature points, yielding the multiplication result for each feature point in the single set: nOffset. Furthermore, the method pBuffer += nOffset is executed to determine the i-th corresponding sub-feature data (pBuffer) from the point set feature data based on the multiplication result (nOffset) of the i-th feature point in the single set of feature points. The memcpy(pBuffer,&szNULL[0],szNULL.size()) method is executed, which resets each element of the two-dimensional array (i.e., sub-feature data (pBuffer)) corresponding to the i-th feature point to zero through an empty array, and hides the two-dimensional array (i.e., sub-feature data (pBuffer)) corresponding to the i-th feature point.

[0103] Step S250: Determine the object detection result corresponding to the image to be detected based on a single set of feature points.

[0104] The object detection result is the detection result of the predetermined object. The object detection result can include the number of predetermined objects detected in the image to be detected, whether the image to be detected contains the predetermined object, the game mode corresponding to the image to be detected, and the interface type corresponding to the image to be detected.

[0105] Each matching of a single set of feature points indicates the detection of a predetermined object from the image to be detected. In one example, the number of predetermined objects detected from the image to be detected can be determined based on the number of matching single sets of feature points. In another example, the number of predetermined objects can be used to further determine whether the game mode is a predetermined mode such as a mixed doubles mode. In yet another example, whether the image to be detected is a predetermined interface type can be determined based on whether a single set of feature points is detected.

[0106] Figure 6 A flowchart illustrating one embodiment of a cloud gaming control method according to this application is shown. The entity executing this cloud gaming control method can be any, such as... Figure 1 The server 110 shown is the target cloud game server, which can be a cloud server.

[0107] like Figure 6 As shown, the cloud gaming control method may include steps S310 to S330.

[0108] Step S310: Acquire the game interface image of the target cloud game; Step S320: Use the game interface image as the image to be detected, perform object detection according to the object detection method of any embodiment of this application, and obtain the object detection result; Step S330: Control the target cloud game according to the object detection result.

[0109] See Figure 7 Server 110 is a cloud server, and terminal 120 is a user terminal. The cloud server can be equipped with boards or containers (also known as cloud gaming hosts) 111. The target cloud game can be installed on the boards or containers (also known as cloud gaming hosts) and run. The terminal 120 can run a cloud gaming client 121. The cloud gaming client can include cloud gaming applications (App) or cloud gaming pages (H5), etc.

[0110] See Figure 8 A cloud gaming control device can run in the board or container (also known as a cloud gaming host) 111. This cloud gaming control device can integrate object detection and game control functions. When the target cloud game starts and during gameplay, the cloud gaming control device can take screenshots of the target cloud game to obtain game interface images. These game interface images are then used as images to be detected. Based on the object detection function, object detection is performed according to any of the aforementioned object detection methods to obtain object detection results. Based on the game control function, the target cloud game is controlled according to the object detection results, such as performing virtual clicks, virtual screen swipes, and actively notifying the cloud gaming client 121 in the terminal 120 to end the game.

[0111] Furthermore, a streaming process can be configured in the board or container (also known as a cloud gaming host) 111. This streaming process can push cloud gaming data, such as sound and game interface images generated by the target cloud game, to the cloud gaming client 121 in the terminal 120 via the streaming server 130. User control events while playing the target cloud game in the cloud gaming client 121 can be transmitted back to the board or container (also known as a cloud gaming host) 111 via the streaming server 130.

[0112] In this way, based on steps S310 to S330, the target cloud game can be controlled efficiently according to the object detection results, effectively accelerating the cloud game process and improving the cloud game experience.

[0113] In one embodiment, before step S320, where the game interface image is used as the image to be detected and object detection is performed according to the object detection method of any embodiment of this application to obtain the object detection result, the method further includes: obtaining a cloud game script file, wherein predetermined detection logic and predetermined control logic are configured in the cloud game script file; step S320, where the game interface image is used as the image to be detected and object detection is performed according to the object detection method of any of the preceding claims to obtain the object detection result, includes: according to the predetermined detection logic, the game interface image is used as the image to be detected and object detection is performed according to the object detection method of any embodiment of this application to obtain the object detection result; step S330, where the target cloud game is controlled according to the object detection result, includes: according to the predetermined control logic, the target cloud game is controlled according to the object detection result.

[0114] The cloud gaming script file is a pre-set detection and control script for the target cloud game. The cloud gaming script file configures pre-set detection logic and pre-set control logic. The pre-set detection logic is a pre-set detection logic that controls the object detection of the game interface image, and the pre-set control logic is a pre-configured control logic that controls the operation process of the cloud game.

[0115] Cloud game script files can be written in languages ​​such as XML or JSON. In one implementation of this example, the cloud game script file is an XML file based on XML. The pre-determined detection logic and pre-determined control logic are both described using XML. Compared to JSON, XML has a clearer logical hierarchy, is easier to modify, and has more standardized configurations such as attribute names.

[0116] The pre-defined detection logic and pre-defined control logic can be nested and combined into a complete cloud game script file according to requirements, using nodes. In one implementation, referring to the table below, the nodes corresponding to the pre-defined detection logic may include match nodes, true / false nodes, and TrueCase / FalseCase nodes; the nodes corresponding to the pre-defined control logic may include action nodes and goto nodes. The function and effect of each node are described in the table below. These nodes can be arranged according to… Figure 10 The nested node structure shown forms the cloud game script file.

[0117]

[0118] See Figure 8 The script acquisition unit in the cloud game control device can acquire cloud game script files. The parsing unit in the logic control module can parse the cloud game script files to obtain the predetermined detection logic and predetermined control logic. The matching unit in the logic control module can perform object detection (i.e., image recognition) on the game interface image of the target cloud game through the object detection function according to the predetermined detection logic. The control unit in the logic control module can control (i.e., manipulate) the target cloud game through the game control function according to the predetermined control logic.

[0119] Specifically, in one approach, the cloud gaming controller is developed using the C++ language. The applicant found that C++ is very suitable for running on the Linux system of the cloud gaming host, offering fast execution speed and low system resource consumption, thus minimizing resource contention with the cloud game. The cloud gaming script file is an XML file based on the XML language. (See also...) Figure 9 The XML file can include "XML description logic and parameters," specifically describing the program's logical structure and setting image matching and other input parameters. The logic control module can perform "C++ interpretation and execution." Within the logic control module, the parsing unit can read and parse the XML script in the XML file to obtain input parameters, pre-defined detection logic, and pre-defined control logic. The matching unit can perform image matching based on the pre-defined detection logic and input parameters, i.e., using object detection functionality to perform object detection (i.e., image recognition) on the target cloud game's interface image to obtain object detection results. The control unit can control the game based on the pre-defined control logic and object detection results, i.e., using game control functions to perform "game control" such as virtual clicks or input information on the cloud game interface.

[0120] In this way, corresponding cloud game script files can be configured for different cloud games, and object detection and cloud game control can be performed flexibly and effectively based on the cloud game script files.

[0121] See one example. Figure 10 and Figure 11 The pre-defined detection logic is as follows: Figure 10 and Figure 11 The node nesting logic is shown below. This predetermined detection logic includes at least one match node. Figure 10 Each match node can be as follows: Figure 11 The process involves an object matching step (such as steps 1, 2, and 3), where `match` nodes can be nested between `true / false` nodes and `TrueCase / FalseCase` nodes. Following this predetermined detection logic, object matching for different objects can be performed at each `match` node. "Object matching" means using the game interface image as the image to be detected and performing object detection according to any of the object detection methods described above.

[0122] If the result of "object matching" in the match node is "match successful (i.e., object detection result successfully obtained)," then the corresponding branch of the true node is followed, and the object detection of the match node after the true node is executed. Each match node can detect different predefined objects, that is, when performing object detection in each match node, matching processing can be performed based on the template feature data corresponding to different predefined objects.

[0123] If the result of "Object Matching" in the match node is "Match Failed (i.e., no object detection result was successfully obtained)," it's possible that an exception occurred in the match node. In this case, you can increase the retry count (incrementing the retry count by 1). If the retry count does not exceed the limit, proceed to the false case node for exception handling (e.g., ...). Figure 11 The exceptions shown (such as exception 1, exception 2, and exception 3) can be handled differently depending on the exception. See also... Figure 11 Each step (i.e., the match node) can be configured with its own unique exception handling (e.g., exception 1 or exception 2), or a common exception handling (e.g., exception 3, such as detecting game announcement pop-ups) can be configured for each step (i.e., the match node). Furthermore, when the number of retries reaches the maximum value configured in the script, the false node will be invoked, at which point object detection fails (i.e., the object detection result is ultimately not obtained successfully). If there is no false node in the pre-defined detection logic, the script execution will be directly reported as failed.

[0124] Understandably, in other examples, the configuration can be tailored to the specific circumstances. Figure 10 and Figure 11 Other pre-defined detection logic besides the example shown.

[0125] In one embodiment, controlling a target cloud game based on object detection results according to predetermined control logic includes: generating control instructions corresponding to the object detection results according to predetermined control logic; and sending the control instructions to the cloud game backend so that the cloud game backend executes the game operation corresponding to the control instructions.

[0126] The predefined control logic can describe the control instructions corresponding to different object detection results, such as clicking the start game control or swiping the screen. Based on the predefined control logic, control instructions corresponding to each object detection result can be generated. These control instructions are then sent to the cloud gaming backend (e.g.,...). Figure 8 The cloud gaming backend can execute game operations corresponding to control commands by using a board or container (also known as a cloud gaming host) 111.

[0127] In one embodiment, acquiring game interface images of a target cloud game includes: if the target cloud game is detected to have entered a specific game scene, acquiring game interface images of the target cloud game in the specific game scene.

[0128] For specific game scenarios, such as cloud game pre-loading scenarios or other predetermined game scenarios, object detection can be performed by capturing game interface images of the target cloud game within the specified game scenario.

[0129] In one example, the specific game scenario is a cloud gaming pre-loading scenario. During the cloud gaming pre-loading phase, refer to... Figures 12 to 13 According to the cloud gaming script file, firstly, following the predetermined detection logic, the following will be performed: Figure 12 The game interface image shown is used as the image to be detected for object detection. The object detection result is "Start Game Control". At this time, the control instruction corresponding to "Start Game Control" can be generated according to the predetermined control logic as "Click Start Game Control". This command is sent to the cloud gaming backend for execution, and the target cloud game enters as shown. Figure 13 The game interface image shown. Then, according to the predetermined detection logic, it can be... Figure 13 The game interface image shown is used as the image to be detected for object detection. The object detection result is "competitive battle control". Then, according to the predetermined control logic, the corresponding control instruction "click competitive battle control" can be generated. This command is sent to the cloud gaming backend for execution, and the target cloud game enters as shown. Figure 14 The game interface image shown. Then, according to the predetermined detection logic, it can be... Figure 14The game interface image shown is used as the image to be detected for object detection. The object detection result is "Human-Machine Mode Entry". Then, according to the predetermined control logic, the control instruction corresponding to "Human-Machine Mode Entry" can be generated as "Click Human-Machine Mode Entry". This command is sent to the cloud gaming backend for execution. The target cloud game enters human-machine mode to complete the cloud game preloading. After that, the cloud gaming backend can send the cloud game data to the user terminal. In this way, the cloud gaming process can be effectively accelerated, user operations can be reduced, and the user experience can be improved.

[0130] The following describes the beneficial effects of the aforementioned embodiments further using an object detection process in an application scenario. (See also...) Figure 15 and Figure 17 The diagram illustrates the interface changes in object detection using related technologies in this scenario. Figure 18 The diagram illustrates the interface changes for object detection using an embodiment of this application in this scenario. Figure 19 The diagram shows a comparison of the process of object detection in related technologies and embodiments of this application in this scenario.

[0131] In this scenario, see Figure 19 In related technologies, the object detection process may include a cyclic process from step S410 to step S440.

[0132] In step S410, feature points are extracted, specifically, using Figure 15 The game interface image is the image to be detected. A batch of feature points is detected from the image to be detected, such as 50 selected feature points. In step S420, modeling is trained, specifically, feature modeling is performed on the detected batch of feature points to generate point set feature data. In step S430, image matching is performed, specifically, object matching is performed based on the point set feature data to obtain the object matching result (a tennis robot). In step S440, image filling is performed, specifically, as shown in the figure. Figure 16 As shown, the region corresponding to the matching result (a tennis robot) is selected from the bounding box of the image to be detected, and the region is filled as shown. Figure 17 The solid color shown; then, with Figure 17 The image shown is used as the image to be detected. Steps S410 to S440 are repeated to perform feature point detection, modeling, matching and filling in a loop to detect objects. The final object detection result is that it contains 3 tennis robots.

[0133] In this scenario, the process of object detection using the embodiments of this application may include steps S510 to S540, with steps S530 to S540 being partially cyclical.

[0134] In step S510, feature points are extracted. Specifically, at least one feature point in the image to be detected is detected, and index information corresponding to each feature point is generated. The image to be detected is as follows: Figure 18 The image on the left, at least one feature point such as Figure 18 As shown in the image on the right; in step S520, training modeling is performed, specifically, feature modeling is performed on at least one feature point to generate point set feature data in the target format; steps S530 to S540 are repeated, in step S530, image matching is performed, specifically, the template feature data and the point set feature data are matched at least once to determine the single set of feature points that match the template feature points from the feature points one by one, and the object detection result corresponding to the image to be detected is determined to contain 3 tennis robots based on the single set of feature points; in step S540, training data is erased, specifically, after each single set of feature points is matched, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format.

[0135] In this scenario, related technologies involve n+1 extractions of feature points and training modeling, and n+1 matching and image filling operations. However, by applying the embodiments of this application, feature point extraction and training modeling are performed only once, followed by n+1 image matching operations and erasure of training data. By applying the embodiments of this application, the computational burden and time consumption for image-based object detection are effectively reduced compared to related technologies.

[0136] To facilitate better implementation of the object detection method and cloud gaming control method provided in the embodiments of this application, the embodiments of this application also provide an object detection device based on the above-described object detection method and a cloud gaming control device based on the above-described cloud gaming control method. The meanings of the terms used are the same as in the above-described object detection method and cloud gaming control method, and specific implementation details can be found in the descriptions in the method embodiments.

[0137] Figure 20 A block diagram of an object detection apparatus according to an embodiment of this application is shown. Figure 21 A block diagram of a cloud gaming control device according to another embodiment of this application is shown.

[0138] like Figure 20 As shown, the object detection device 600 may include an acquisition module 610, a detection module 620, a modeling module 630, a matching module 640, and a determination module 650.

[0139] The acquisition module 610 can be used to acquire template feature data of template feature points of the image to be detected and the predetermined object; the detection module 620 can be used to detect at least one feature point included in the image to be detected and generate index information corresponding to each feature point; the modeling module 630 can be used to perform feature modeling on the at least one feature point and generate point set feature data in the target format; the matching module 640 can be used to perform at least one matching process between the template feature data and the point set feature data to determine the single set of feature points that match the template feature points from the feature points one by one, wherein, after each matching of a single set of feature points, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format; the determination module 650 can be used to determine the object detection result corresponding to the image to be detected based on the single set of feature points.

[0140] In some embodiments of this application, the matching module includes: a matching unit, configured to perform matching processing on the template feature data and the point set feature data to obtain a single set of feature points that match the template feature points among the at least one feature points, wherein the single set of feature points is a feature point corresponding to a single predetermined object; a positioning unit, configured to determine sub-feature data corresponding to the single set of feature points in the point set feature data according to the index information and the target format corresponding to the single set of feature points; a hiding unit, configured to perform hiding processing on the sub-feature data in the point set feature data to obtain hidden feature data; and a repeating unit, configured to replace the point set feature data with the hidden feature data and repeatedly execute the above matching processing to hiding processing steps to obtain at least one set of single feature points that match the template feature points.

[0141] In some embodiments of this application, the index information is a number, and the target format point set feature data is generated by concatenating at least one two-dimensional array of target size according to the number of feature points, with each image channel of the feature point corresponding to one two-dimensional array; the positioning unit is used to: multiply the number corresponding to each feature point in the single set of feature points, the target size, and the number of image channels to obtain the multiplication result corresponding to each feature point in the single set of feature points; and determine the sub-feature data corresponding to each feature point in the single set of feature points from the point set feature data based on the multiplication result corresponding to each feature point in the single set of feature points.

[0142] In some embodiments of this application, the hiding unit is used to: reset the sub-feature data in the point set feature data to predetermined data to obtain the hidden feature data, wherein the predetermined data includes either zero or infinity.

[0143] In some embodiments of this application, the modeling module includes: a neighborhood determination unit, configured to select a neighborhood corresponding to each feature point in the image to be detected for the at least one feature point; a partitioning unit, configured to divide the neighborhood corresponding to each feature point into a predetermined number of sub-domains, and perform feature extraction processing on the sub-domains respectively to obtain a predetermined number of sub-domain feature data corresponding to each feature point; and a generation unit, configured to generate point set feature data in the target format based on the predetermined number of sub-domain feature data corresponding to each feature point.

[0144] In some embodiments of this application, the target format point set feature data is generated based on at least one two-dimensional array of target size, and each two-dimensional array corresponds to one feature point; the generation unit is configured to: take a predetermined number of subdomain feature data corresponding to each feature point as array elements to generate a two-dimensional array of target size corresponding to each feature point; and generate the target format point set feature data based on the two-dimensional array of target size corresponding to each feature point.

[0145] like Figure 21 As shown, a cloud gaming control device 700 includes: a data acquisition unit 710 for acquiring game interface images of a target cloud game; a matching unit 720 for using the game interface images as images to be detected and performing object detection according to the object detection method described in any embodiment of this application to obtain object detection results; and a control unit 730 for controlling the target cloud game based on the object detection results.

[0146] In some embodiments of this application, the cloud gaming control device further includes: a script acquisition unit, configured to: acquire a cloud gaming script file, wherein the cloud gaming script file is configured with predetermined detection logic and predetermined control logic; a matching unit, configured to: according to the predetermined detection logic, take the game interface image as the image to be detected, perform object detection according to the object detection method described in any embodiment of this application, and obtain an object detection result; and a control unit, configured to: according to the predetermined control logic, control the target cloud game based on the object detection result.

[0147] In some embodiments of this application, the control unit is configured to: generate a control instruction corresponding to the object detection result according to the predetermined control logic; and send the control instruction to the cloud gaming backend so that the cloud gaming backend executes the game operation corresponding to the control instruction.

[0148] In some embodiments of this application, the acquisition unit is configured to: if the target cloud game is detected to have entered a specific game scene, acquire the game interface image of the target cloud game in the specific game scene.

[0149] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0150] Furthermore, embodiments of this application also provide an electronic device, which can be a terminal or a server, such as... Figure 22 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0151] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 22 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0152] The processor 801 is the control center of the electronic device. It connects to various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions of the computer device and processes data, thereby detecting the electronic device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0153] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0154] The electronic device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0155] The electronic device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0156] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 802 according to the following instructions, and the processor 801 runs the computer programs stored in the memory 802, thereby realizing the various functions of the foregoing embodiments of this application.

[0157] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0158] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program that can be loaded by a processor to perform the steps in any of the methods provided in embodiments of this application.

[0159] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc. Since the computer program stored in the computer-readable storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects achievable by the methods provided in the embodiments of this application can be realized. See the preceding embodiments for details, which will not be repeated here.

[0160] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments of this application.

[0161] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0162] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. An object detection method, characterized in that, include: Obtain template feature data of template feature points of the image to be detected and the predetermined object; Detect at least one feature point in the image to be detected, and generate index information corresponding to each feature point; wherein, the index information is a number; Feature modeling is performed on the at least one feature point to generate point set feature data in a target format; wherein, the point set feature data in the target format is generated by concatenating at least one two-dimensional array of target size according to the number of the feature points, and each image channel of the feature point corresponds to one of the two-dimensional arrays; The template feature data and the point set feature data are matched at least once to determine the single set of feature points that match the template feature points from the feature points one by one. Each time a single set of feature points is matched, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and target format. The single set of feature points is the feature point corresponding to a single predetermined object. The object detection result corresponding to the image to be detected is determined based on the single set of feature points.

2. The method according to claim 1, characterized in that, The step of performing at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points matching the template feature points from the feature points includes: The template feature data is matched with the point set feature data to obtain a single set of feature points that match the template feature points in the at least one feature point; Based on the index information and target format corresponding to the single set of feature points, determine the sub-feature data corresponding to the single set of feature points in the point set feature data; The sub-feature data in the point set feature data is hidden to obtain the hidden feature data; The hidden feature data is used to replace the point set feature data, and the above matching and hiding processes are repeated to obtain at least one set of single feature points that match the template feature points.

3. The method according to claim 2, characterized in that, The step of determining the sub-feature data corresponding to the single set of feature points in the point set feature data based on the index information and the target format corresponding to the single set of feature points includes: Multiply the number corresponding to each feature point in the single group of feature points, the target size, and the number of image channels to obtain the multiplication result corresponding to each feature point in the single group of feature points; Based on the multiplication result corresponding to each feature point in the single set of feature points, the sub-feature data corresponding to each feature point in the single set of feature points is determined from the feature data of the point set.

4. The method according to claim 2, characterized in that, The step of hiding the sub-feature data in the point set feature data to obtain hidden feature data includes: The sub-feature data in the point set feature data is reset to predetermined data to obtain the hidden feature data, wherein the predetermined data includes either zero or infinity.

5. The method according to claim 1, characterized in that, The step of performing feature modeling on the at least one feature point to generate point set feature data in the target format includes: For each feature point, a neighborhood corresponding to each feature point is selected in the image to be detected; The neighborhood corresponding to each feature point is divided into a predetermined number of subdomains, and feature extraction processing is performed on each subdomain to obtain feature data of the predetermined number of subdomains corresponding to each feature point. Based on a predetermined number of subdomain feature data corresponding to each feature point, point set feature data in the target format is generated.

6. The method according to claim 5, characterized in that, The target format point set feature data is generated based on a two-dimensional array of at least one target size, and each two-dimensional array corresponds to one feature point; The step of generating the point set feature data in the target format based on a predetermined number of sub-domain feature data corresponding to each feature point includes: A predetermined number of subdomain feature data corresponding to each feature point are used as array elements to generate a two-dimensional array of target size corresponding to each feature point; Based on a two-dimensional array of target size corresponding to each feature point, point set feature data in the target format is generated.

7. A cloud gaming control method, characterized in that, include: Capture images of the target cloud game's interface; Using the game interface image as the image to be detected, object detection is performed according to the method described in any one of claims 1 to 6 to obtain the object detection result; The target cloud game is controlled based on the object detection results.

8. The method according to claim 7, characterized in that, Before taking the game interface image as the image to be detected and performing object detection according to the method of any one of claims 1 to 6 to obtain the object detection result, the method further includes: Obtain a cloud game script file, wherein the cloud game script file is configured with pre-defined detection logic and pre-defined control logic; The step of using the game interface image as the image to be detected and performing object detection according to the method of any one of claims 1 to 6 to obtain the object detection result includes: According to the predetermined detection logic, the game interface image is taken as the image to be detected, and object detection is performed according to the method of any one of claims 1 to 6 to obtain the object detection result; The step of controlling the target cloud game based on the object detection result includes: The target cloud game is controlled according to the predetermined control logic and the object detection result.

9. The method according to claim 8, characterized in that, The step of controlling the target cloud game according to the predetermined control logic and the object detection result includes: According to the predetermined control logic, control instructions corresponding to the object detection results are generated; The control command is sent to the cloud gaming backend so that the cloud gaming backend executes the game operation corresponding to the control command.

10. The method according to claim 7, characterized in that, The collected game interface images of the target cloud game include: If the target cloud game is detected to have entered a specific game scene, the game interface image of the target cloud game in the specific game scene is captured.

11. An object detection device, characterized in that, include: The acquisition module is used to acquire template feature data of template feature points of the image to be detected and the predetermined object; The detection module is used to detect at least one feature point included in the image to be detected, and generate index information corresponding to each feature point; wherein, the index information is a number; A modeling module is used to perform feature modeling on the at least one feature point and generate point set feature data in a target format; wherein, the point set feature data in the target format is generated by concatenating at least one two-dimensional array of target size according to the number of the feature points, and each image channel of the feature point corresponds to one two-dimensional array; The matching module is used to perform at least one matching process between the template feature data and the point set feature data to successively determine a single set of feature points that match the template feature points from the feature points. Each time a single set of feature points is matched, the sub-feature data corresponding to the single set of feature points in the point set feature data is hidden according to the index information and the target format. The single set of feature points is the feature point corresponding to a single predetermined object. The determination module is used to determine the object detection result corresponding to the image to be detected based on the single set of feature points.

12. The apparatus according to claim 11, characterized in that, The matching module includes: A matching unit is used to match the template feature data with the point set feature data to obtain a single set of feature points that match the template feature points in the at least one feature point. A localization unit is used to determine the sub-feature data corresponding to the single set of feature points in the point set feature data based on the index information and the target format corresponding to the single set of feature points. A hiding unit is used to hide the sub-feature data in the point set feature data to obtain the hidden feature data; The repeating unit is used to replace the point set feature data with the hidden feature data and repeatedly execute the above matching process to the hiding process to obtain at least one set of single feature points that match the template feature points.

13. The apparatus according to claim 12, characterized in that, The positioning unit is configured to: multiply the number corresponding to each feature point in the single set of feature points, the target size, and the number of image channels to obtain the multiplication result corresponding to each feature point in the single set of feature points; and determine the sub-feature data corresponding to each feature point in the single set of feature points from the point set feature data based on the multiplication result corresponding to each feature point in the single set of feature points.

14. The apparatus according to claim 12, characterized in that, The hiding unit is used to: reset the sub-feature data in the point set feature data to predetermined data to obtain the hidden feature data, wherein the predetermined data includes either zero or infinity.

15. The apparatus according to claim 11, characterized in that, The modeling module includes: A neighborhood determination unit is used to select a neighborhood corresponding to each feature point in the image to be detected, for the at least one feature point. A partitioning unit is used to divide the neighborhood corresponding to each feature point into a predetermined number of subdomains, and to perform feature extraction processing on each subdomain to obtain feature data of the predetermined number of subdomains corresponding to each feature point. The generation unit is used to generate point set feature data in the target format based on a predetermined number of subdomain feature data corresponding to each feature point.

16. The apparatus according to claim 15, characterized in that, The target format point set feature data is generated based on a two-dimensional array of at least one target size, and each two-dimensional array corresponds to one feature point; The generation unit is configured to: use a predetermined number of subdomain feature data corresponding to each feature point as array elements to generate a two-dimensional array of target size corresponding to each feature point; and generate point set feature data of the target format based on the two-dimensional array of target size corresponding to each feature point.

17. A cloud gaming control device, characterized in that, include: The acquisition unit is used to acquire images of the game interface of the target cloud game. A matching unit is used to take the game interface image as the image to be detected, and perform object detection according to the method of any one of claims 1 to 6 to obtain the object detection result; A control unit is used to control the target cloud game based on the object detection results.

18. The apparatus according to claim 17, characterized in that, The cloud gaming control device also includes: The script acquisition unit is used to: acquire a cloud game script file, wherein the cloud game script file is configured with predetermined detection logic and predetermined control logic; The matching unit is configured to: take the game interface image as the image to be detected according to the predetermined detection logic, perform object detection according to the object detection method according to any one of claims 1 to 6, and obtain the object detection result; The control unit is configured to: control the target cloud game according to the predetermined control logic and the object detection result.

19. The apparatus according to claim 18, characterized in that, The control unit is configured to: generate a control instruction corresponding to the object detection result according to the predetermined control logic; and send the control instruction to the cloud gaming backend so that the cloud gaming backend executes the game operation corresponding to the control instruction.

20. The apparatus according to claim 17, characterized in that, The acquisition unit is used to: if the target cloud game is detected to have entered a specific game scene, acquire the game interface image of the target cloud game in the specific game scene.

21. A computer-readable storage medium, characterized in that, It contains a computer program that, when executed by the computer's processor, causes the computer to perform the method described in any one of claims 1 to 6 or claims 7 to 10.

22. An electronic device, characterized in that, include: Memory, which stores computer programs; A processor reads a computer program stored in memory to perform the method described in any one of claims 1 to 6 or 7 to 10.

23. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6 or claims 7 to 10.

Images