Target detection method, device, equipment, readable storage medium and program product

Abstract

Embodiments of the present application provide a target detection method and device, equipment, readable storage medium and program product, which are related to the fields of cloud computing, cloud game, map, etc., and the application scenarios include but are not limited to game scenarios. The method comprises: acquiring a target image and a reference image, the target image being a next frame image adjacent to the reference image; determining a pixel spectrum of each target object in the target image and predicted position information of each target object; determining a predicted identifier of each target object based on the reference image; correcting the predicted identifier of each target object and the predicted position information of each target object based on the pixel spectrum of each target object, the predicted position information of each target object, a pixel spectrum of a plurality of reference objects in the reference image and position information of the plurality of reference objects, to determine an identifier of each target object and position information of each target object; in this way, the detection success rate of the plurality of target objects in the target image is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a target detection method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] In games, target detection of game characters is necessary; the game characters themselves are the target objects. In complex, large-scale games, with continuous scene changes and version updates, more and more target objects of varying sizes will appear. Current technologies use deep learning-based or frame difference-based target detection algorithms to detect multiple target objects in games; however, these algorithms are not very effective. For example, when multiple target objects move slowly, missed detections are common, and overlapping areas between multiple target objects may go undetected, resulting in voids or other artifacts. Therefore, the success rate of detecting multiple target objects in games is low. Summary of the Invention

[0003] This application addresses the shortcomings of existing methods by proposing a target detection method, apparatus, device, computer-readable storage medium, and computer program product to solve the problem of how to improve the detection success rate of multiple target objects.

[0004] Firstly, this application provides a target detection method, including:

[0005] Acquire the target image and the reference image, where the target image is the next frame image adjacent to the reference image;

[0006] Determine the pixel spectrum of each target object in the target image and the predicted location information of each target object;

[0007] Based on the reference image, determine the predicted label for each target object;

[0008] Based on the pixel spectrum of each target object, the predicted location information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the location information of multiple reference objects, the predicted identifier and predicted location information of each target object are corrected to determine the identifier and location information of each target object.

[0009] In one embodiment, determining the pixel spectrum of each target object in the target image and the predicted location information of each target object includes:

[0010] Determine the feature vector of the target image;

[0011] Based on the feature vector of the target image, determine the pixel spectrum of each target object in the target image;

[0012] Based on the pixel spectrum of each target object, the pixels in the target image are classified by a preset pixel-level classifier to determine the predicted position information of each target object. The predicted position information is the coordinates of the center pixel of each target object.

[0013] In one embodiment, determining the feature vector of the target image includes:

[0014] The target image is convolved with each convolution kernel in the preset convolutional layer to obtain the sub-vector features corresponding to each convolution kernel. The convolutional layer includes multiple convolution kernels.

[0015] The feature vectors of the target image are obtained by concatenating the features of each sub-vector.

[0016] In one embodiment, determining the pixel spectrum of each target object in the target image based on the feature vector of the target image includes:

[0017] The feature vector of the target image is reduced in dimension to obtain the reduced feature vector.

[0018] The reduced feature vectors are then smoothed and subjected to Fourier transform to obtain the pixel spectrum of each target object in the target image.

[0019] In one embodiment, determining the predicted identifier for each target object based on a reference image includes:

[0020] Based on the amplitude and phase of the center pixel of any reference object in the reference image, determine the corresponding position of the center pixel of any reference object in the target image;

[0021] If the center pixel of each target object is in the corresponding position, then the identifier of any reference object is determined as the predicted identifier of each target object.

[0022] In one embodiment, based on the pixel spectrum of each target object, the predicted location information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the location information of multiple reference objects, the predicted identifier and predicted location information of each target object are corrected to ensure the identifier and location information of each target object, including:

[0023] Determine the pixel spectral amplitude and pixel spectral phase of each target object, as well as the pixel spectral amplitude and pixel spectral phase of multiple reference objects;

[0024] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of multiple reference objects, the predicted label of each target object is corrected to determine the label of each target object.

[0025] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the predicted position information of each target object is corrected to determine the position information of each target object; multiple reference objects include the reference object corresponding to the identifier.

[0026] In one embodiment, determining the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of multiple reference objects, and the pixel spectral phase of multiple reference objects includes:

[0027] By using Fourier transform, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the pixel spectrum of each target object, and the pixel spectrum amplitude and pixel spectrum phase of multiple reference objects are extracted from the pixel spectra of multiple reference objects.

[0028] In one embodiment, the predicted identifier of each target object is corrected based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of multiple reference objects, and the pixel spectral phase of multiple reference objects to determine the identifier of each target object, including:

[0029] Determine the difference between the pixel spectral amplitude of each target object and the pixel spectral amplitude of the reference object corresponding to the predicted label, where multiple reference objects include the reference object corresponding to the predicted label;

[0030] If the difference is greater than the preset threshold, the predicted label of each target object is corrected based on the pixel spectrum amplitude of each target object, the pixel spectrum phase of each target object, the pixel spectrum amplitude of other reference objects among multiple reference objects, and the pixel spectrum phase of other reference objects, and the label of each target object is determined; other reference objects do not include the reference object corresponding to the predicted label.

[0031] In one embodiment, the predicted identifier of each target object is corrected based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of other reference objects among multiple reference objects, and the pixel spectral phase of other reference objects, to determine the identifier of each target object, including:

[0032] Based on the pixel spectrum amplitude and pixel spectrum phase of each target object, as well as the pixel spectrum amplitude and pixel spectrum phase of other reference objects among multiple reference objects, the probabilistic similarity between each target object and each other reference object is determined, and the identifier of the reference object corresponding to the highest probability similarity among all probability similarities is determined as the identifier of each target object.

[0033] In one embodiment, the predicted position information of each target object is corrected based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude of the corresponding reference object, and the pixel spectral phase of the corresponding reference object, to determine the position information of each target object, including:

[0034] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the expected deviation value of the predicted position information of each target object is determined.

[0035] The location information of each target object is determined based on the predicted location information and the expected deviation value.

[0036] Secondly, this application provides a target detection device, comprising:

[0037] The first processing module is used to acquire a target image and a reference image, wherein the target image is the next frame image adjacent to the reference image;

[0038] The second processing module is used to determine the pixel spectrum of each target object in the target image and the predicted location information of each target object.

[0039] The third processing module is used to determine the predicted identifier of each target object based on the reference image;

[0040] The fourth processing module is used to correct the predicted identifier and predicted position information of each target object based on the pixel spectrum of each target object, the predicted position information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the position information of multiple reference objects, and to determine the identifier and position information of each target object.

[0041] Thirdly, this application provides an electronic device, including: a processor, a memory, and a bus;

[0042] A bus is used to connect the processor and memory;

[0043] Memory, used to store operation instructions;

[0044] The processor is used to execute the target detection method of the first aspect of this application by invoking operation instructions.

[0045] Fourthly, this application provides a computer-readable storage medium storing a computer program that is used to perform the target detection method of the first aspect of this application.

[0046] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the target detection method in the first aspect of this application.

[0047] The technical solution provided in this application has at least the following beneficial effects:

[0048] The process involves acquiring a target image and a reference image. The target image is the next frame adjacent to the reference image, i.e., the target image is the image of frame t, and the reference image is the image of frame t-1. The process then involves determining the pixel spectrum and predicted location information of each target object in the target image. Based on the reference image, a predicted identifier for each target object is determined. Based on the pixel spectrum and predicted location information of each target object, as well as the pixel spectrum and location information of multiple reference objects in the reference image, the predicted identifier and predicted location information of each target object are corrected, and the identifier and location information of each target object are determined. This process is repeated to separate the target image in frame t, obtaining the pixel spectrum and predicted location information (predicted location information) of each target object in frame t. Each target object in frame t is then assigned an ID, i.e., the ID (predicted identifier) ​​of each target object in frame t is predicted. Finally, ID correction and predicted location information correction are performed for each target object in frame t. This improves the detection success rate of each target object in the target image, thus improving the detection success rate of multiple target objects in the target image. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0050] Figure 1 This is a schematic diagram of the architecture of the target detection system provided in the embodiments of this application;

[0051] Figure 2 A schematic flowchart of a target detection method provided in an embodiment of this application;

[0052] Figure 3 A schematic flowchart of a target detection method provided in an embodiment of this application;

[0053] Figure 4A schematic diagram illustrating target detection provided in an embodiment of this application;

[0054] Figure 5 A schematic diagram illustrating target detection provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application;

[0056] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0057] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0058] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0059] It is understood that in the specific embodiments of this application, data related to target detection is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0061] This application provides an example of a target detection method for a target detection system, which relates to fields such as cloud computing, cloud gaming, and mapping.

[0062] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0063] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0064] Intelligent Traffic Systems (ITS), also known as Intelligent Transportation Systems, effectively integrate advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing. This strengthens the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, enhances the environment, and saves energy.

[0065] To better understand and explain the solutions of the embodiments of this application, some technical terms involved in the embodiments of this application will be briefly explained below.

[0066] Fourier Transform: The Fourier Transform (FFT) represents a function that satisfies certain conditions as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. The Fourier Transform has many different variants, such as the continuous Fourier Transform and the discrete Fourier Transform.

[0067] Pixel spectrum: The pixel spectrum is the frequency domain representation of an image, representing the proportion of pixels in different frequency domains. The pixel spectrum is also a matrix.

[0068] RGB: The RGB color mode is an industry color standard that uses variations in the three color channels—red (R), green (G), and blue (B)—and their superposition to obtain a variety of colors. RGB represents the colors of the three channels: red, green, and blue.

[0069] MOBA games: MOBA (Multiplayer Online Battle Arena) games are action-based real-time strategy games.

[0070] The solutions provided in this application relate to cloud computing technology. The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0071] To better understand the solution provided in the embodiments of this application, the solution will be described below in conjunction with a specific application scenario.

[0072] In one embodiment, Figure 1 The diagram shows an architecture schematic of a target detection system applicable to embodiments of this application. It is understood that the target detection method provided in these embodiments can be applied to, but is not limited to, applications such as... Figure 1 In the application scenarios shown.

[0073] In this example, as Figure 1 As shown, the architecture of the target detection system in this example may include, but is not limited to, server 10, terminal 20, and database 30. Server 10, terminal 20, and database 30 can interact with each other via network 40.

[0074] Server 10 acquires a target image and a reference image, where the target image is the next frame adjacent to the reference image. Server 10 determines the pixel spectrum and predicted position information of each target object in the target image. Based on the reference image, server 10 determines the predicted identifier of each target object. Based on the pixel spectrum and predicted position information of each target object, as well as the pixel spectrum and position information of multiple reference objects in the reference image, server 10 corrects the predicted identifier and predicted position information of each target object to determine the identifier and position information of each target object. Server 10 sends the identifier and position information of each target object to terminal 20 via network 40. Terminal 20 displays each target object and its identifier at the corresponding position. Each target object can be identified by a detection box, a numerical marker, etc. For example, different target objects can be identified by detection boxes of different colors, with target object A identified by a red detection box, target object B by a blue detection box, and target object C by a yellow detection box. Alternatively, different target objects can be identified by different numerical markers, with target object A identified by 1, target object B by 2, and target object C by 3. Server 10 sends the identifier of each target object and the location information of each target object to database 30 for storage via network 40.

[0075] It is understood that the above is only one example, and this embodiment is not limited here.

[0076] Terminals include, but are not limited to, smartphones (such as Android phones, iOS phones, etc.), mobile phone emulators, tablets, laptops, digital broadcast receivers, MIDs (Mobile Internet Devices), PDAs (Personal Digital Assistants), smart voice interaction devices, smart home appliances, and in-vehicle terminals.

[0077] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0078] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0079] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0080] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

[0081] Artificial intelligence cloud services are generally also known as AIaaS (AI as a Service). This is currently a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.

[0082] The aforementioned networks may include, but are not limited to, wired networks and wireless networks. Wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. Specific details can be determined based on actual application scenario requirements and are not limited here.

[0083] See Figure 2 , Figure 2 This illustration shows a flowchart of a target detection method provided in an embodiment of this application. The method can be executed by any electronic device, such as a terminal or server. As an optional implementation, the method can be executed by a server. For ease of description, the following descriptions of some optional embodiments will use a server as the execution subject of the method. Figure 2 As shown, the target detection method provided in this application includes the following steps:

[0084] S201, acquire the target image and the reference image, where the target image is the next frame image adjacent to the reference image.

[0085] Specifically, the target image is the image at frame t in the game, and the reference image is the image at frame t-1 in the game. The target image contains multiple game characters, and the reference image also contains multiple game characters.

[0086] S202, determine the pixel spectrum of each target object in the target image and the predicted location information of each target object.

[0087] Specifically, the target object can be a game character in a game; there are multiple target objects in the target image, meaning there are multiple game characters in the target image. The predicted position information of the target object can be the coordinates of the center position of the target object, that is, the coordinates of the center pixel of the target object; due to the influence of noise, the coordinates of the center position of the target object may be deviated, therefore, it is necessary to correct the coordinates of the center position of the target object, that is, to correct the predicted position information of the target object.

[0088] S203, Based on the reference image, determine the predicted identifier for each target object.

[0089] Specifically, the predicted identifier for each target object can be a detection box, a numerical marker, etc. For example, the predicted identifiers for different target objects can be detection boxes of different colors, with the predicted identifier for target object A being a red detection box, the predicted identifier for target object B being a blue detection box, and the predicted identifier for target object C being a yellow detection box. As another example, the predicted identifiers for different target objects can be different numerical markers, with the predicted identifier for target object A being 1, the predicted identifier for target object B being 2, and the predicted identifier for target object C being 3.

[0090] S204, based on the pixel spectrum of each target object, the predicted position information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the position information of multiple reference objects, correct the predicted identifier and predicted position information of each target object, and determine the identifier and position information of each target object.

[0091] Specifically, each target object corresponds to a detection bucket, which includes the pixel spectrum of the target object and its predicted position information. Each reference object corresponds to a detection bucket, which includes the pixel spectrum of the reference object and its position information. The predicted identifier and predicted position information of a target object are corrected to determine the target object's identifier and position information; the predicted position information of the target object in its corresponding detection bucket is then updated to reflect the target object's actual position.

[0092] In this embodiment, a target image and a reference image are acquired. The target image is the next frame adjacent to the reference image, i.e., the target image is the image of frame t, and the reference image is the image of frame t-1. The pixel spectrum and predicted position information of each target object in the target image are determined. Based on the reference image, the predicted identifier of each target object is determined. Based on the pixel spectrum and predicted position information of each target object, the pixel spectrum and position information of multiple reference objects in the reference image, the predicted identifier and predicted position information of each target object are corrected to determine the identifier and position information of each target object. Thus, target separation is performed on the target image of frame t to obtain the pixel spectrum and predicted position information (predicted position information) of each target object in frame t. Each target object in frame t is ID-labeled, i.e., the ID (predicted identifier) ​​of each target object in frame t is predicted. The ID correction and predicted position information correction of each target object in frame t are performed. This improves the detection success rate of each target object in the target image, i.e., improves the detection success rate of multiple target objects in the target image.

[0093] In one embodiment, determining the pixel spectrum of each target object in the target image and the predicted location information of each target object includes:

[0094] Determine the feature vector of the target image;

[0095] Based on the feature vector of the target image, determine the pixel spectrum of each target object in the target image;

[0096] Based on the pixel spectrum of each target object, the pixels in the target image are classified by a preset pixel-level classifier to determine the predicted position information of each target object. The predicted position information is the coordinates of the center pixel of each target object.

[0097] Specifically, for example, the target image is convolved with each of the 12 convolutional kernels in a predefined convolutional layer (3×3, 12), resulting in sub-vector features corresponding to each of the 12 kernels. Here, 3×3 in the convolutional layer (3×3, 12) represents the size of the convolutional kernel, i.e., the height × width of the kernel; 12 represents the dimension, meaning the convolutional layer has 12 3×3 convolutional kernels. Different convolutional kernels produce different image information, therefore, different kernels correspond to different sub-vector features. The main difference between the kernels lies in their frequency; for example, the 12 kernels include 3 high-frequency kernels, 3 low-frequency kernels, and 6 mid-to-high-frequency kernels. The image information produced by the high-frequency kernels can be the edge parts of the image, such as contour lines; the image information produced by the low-frequency kernels can be the smooth, detailed areas of the image, such as faces; and the image information produced by the mid-to-high-frequency kernels can be the middle parts of the image, such as gradually changing areas. The sub-vector features corresponding to the 12 convolution kernels are concatenated to obtain the feature vector of the target image. The feature vector of the target image can be a clustered convolution matrix. Among them, the 3×3 matrix generated by the arctangent function arctan can be used as the 3×3 convolution kernel.

[0098] The feature vector of the target image is reduced in dimensionality by a pooling layer (e.g., max pooling layer) to obtain the dimensionality-reduced feature vector. The dimensionality-reduced feature vector is then convolved with a 3×3 convolution kernel with smoothing filtering and subjected to Fourier transform to obtain the pixel spectrum of each target object in the target image. The pixel spectrum of each target object can be an RGB color pixel map.

[0099] The pixel spectrum of each target object and the pixels in the target image are input into a preset pixel-level classifier. The pixel-level classifier classifies the pixels in the target image based on the pixel spectrum of each target object and determines the coordinates of the center position pixel of each target object. In this way, target separation in the target image of the t-th frame is achieved, that is, the target object and the background in the target image are distinguished.

[0100] In one embodiment, determining the feature vector of the target image includes:

[0101] The target image is convolved with each convolution kernel in the preset convolutional layer to obtain the sub-vector features corresponding to each convolution kernel. The convolutional layer includes multiple convolution kernels.

[0102] The feature vectors of the target image are obtained by concatenating the features of each sub-vector.

[0103] Specifically, for example, the target image is convolved with each of the 12 convolutional kernels in a predefined convolutional layer (3×3, 12), resulting in sub-vector features corresponding to each of the 12 kernels. Here, 3×3 in the convolutional layer (3×3, 12) represents the size of the convolutional kernel, i.e., the height × width of the kernel; 12 represents the dimension, meaning the convolutional layer has 12 3×3 convolutional kernels. Different convolutional kernels produce different image information, therefore, different kernels correspond to different sub-vector features. The main difference between the kernels lies in their frequency; for example, the 12 kernels include 3 high-frequency kernels, 3 low-frequency kernels, and 6 mid-to-high-frequency kernels. The image information produced by the high-frequency kernels can be the edge parts of the image, such as contour lines; the image information produced by the low-frequency kernels can be the smooth, detailed areas of the image, such as faces; and the image information produced by the mid-to-high-frequency kernels can be the middle parts of the image, such as gradually changing areas.

[0104] The sub-vector features corresponding to the 12 convolution kernels are concatenated to obtain the feature vector of the target image. The feature vector of the target image can be a clustered convolution matrix. Among them, the 3×3 matrix generated by the arctangent function arctan can be used as the 3×3 convolution kernel.

[0105] In one embodiment, determining the pixel spectrum of each target object in the target image based on the feature vector of the target image includes:

[0106] The feature vector of the target image is reduced in dimension to obtain the reduced feature vector.

[0107] The reduced feature vectors are then smoothed and subjected to Fourier transform to obtain the pixel spectrum of each target object in the target image.

[0108] Specifically, the feature vector of the target image is dimensionality reduced by a pooling layer (e.g., max pooling) to obtain the dimensionality-reduced feature vector; the dimensionality-reduced feature vector is convolved with a 3×3 convolution kernel with smoothing filtering function and then subjected to Fourier transform to obtain the pixel spectrum of each target object in the target image; wherein, the pixel spectrum of each target object can be a pixel map of RGB color.

[0109] In one embodiment, the pixel spectrum of each target object is normalized and upsampled to obtain the processed pixel spectrum of each target object; the processed pixel spectrum of each target object and the pixels in the target image are input to a preset pixel-level classifier, which classifies the pixels in the target image based on the processed pixel spectrum of each target object and determines the coordinates of the center position pixel of each target object.

[0110] In one embodiment, determining the predicted identifier for each target object based on a reference image includes:

[0111] Based on the amplitude and phase of the center pixel of any reference object in the reference image, determine the corresponding position of the center pixel of any reference object in the target image;

[0112] If the center pixel of each target object is in the corresponding position, then the identifier of any reference object is determined as the predicted identifier of each target object.

[0113] Specifically, for example, based on the amplitude and phase of the center pixel B of reference object A in the reference image of frame t-1, the amplitude and phase of the center pixel B in the target image of frame t are determined; based on the amplitude, phase, amplitude, and phase of the center pixel B in frame t-1, the movement deviation of the center pixel B between frame t-1 and frame t is determined, obtaining the motion trajectory of the center pixel B from position M in frame t-1 to position N in frame t, thereby predicting the corresponding position of the center pixel B of reference object A in the target image. If the center pixel of a target object is at the corresponding position, the identifier of reference object A can be predicted as the predicted identifier of the target object; for example, if the identifier of reference object A is 5, the predicted identifier of the target object at the corresponding position can be predicted as 5, that is, the target object and reference object A are predicted to be the same game character.

[0114] In one embodiment, based on the pixel spectrum of each target object, the predicted location information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the location information of multiple reference objects, the predicted identifier and predicted location information of each target object are corrected to confirm the identifier and location information of each target object, including steps A1-A3:

[0115] Step A1: Determine the pixel spectral amplitude and pixel spectral phase of each target object, as well as the pixel spectral amplitude and pixel spectral phase of multiple reference objects.

[0116] Specifically, through Fourier transform, the pixel spectrum amplitude and phase of a target object are extracted from its pixel spectrum. This involves segmenting the pixel spectrum amplitude and phase to obtain the pixel spectrum amplitude and phase of the target object. Similarly, through Fourier transform, the pixel spectrum amplitude and phase of a reference object are extracted from its pixel spectrum. This involves segmenting the pixel spectrum amplitude and phase of the reference object to obtain the pixel spectrum amplitude and phase of the reference object.

[0117] Step A2: Based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of multiple reference objects, and the pixel spectral phase of multiple reference objects, the predicted identifier of each target object is corrected to determine the identifier of each target object.

[0118] Specifically, for example, if the predicted identifier of a target object A is 9, and the difference between the pixel spectrum amplitude of the target object A and the pixel spectrum amplitude of a reference object B (the reference object B is identified as 9) is greater than a preset threshold, then it is determined that the target object A and the reference object B are not the same game character, that is, the predicted identifier 9 of the target object A is inaccurate and the predicted identifier 9 of the target object A needs to be corrected.

[0119] Based on the pixel spectral amplitude and pixel spectral phase of target object A, the pixel spectral amplitude and pixel spectral phase of other reference objects besides reference object B, the predicted identifier 9 of target object A is corrected to determine the identifier of target object A. For example, if the difference between the pixel spectral amplitude of target object A and the pixel spectral amplitude of reference object C (identified as 5) is less than or equal to a preset threshold, the difference between the pixel spectral amplitude of target object A and the pixel spectral amplitude of reference object D (identified as 6) is less than or equal to a preset threshold, and the difference between the pixel spectral amplitude of target object A and the pixel spectral amplitude of reference object E (identified as 7) is less than or equal to a preset threshold, then the target is determined by calculating the Gaussian mixture corrector based on formula (1). The probability similarity φ1 between object A and reference object C, the probability similarity φ2 between target object A and reference object D, and the probability similarity φ3 between target object A and reference object E; the identifier of the reference object corresponding to the highest probability similarity among φ1, φ2, and φ3 is determined as the identifier of target object A. For example, if φ1 is the highest probability similarity among φ1, φ2, and φ3, then the identifier 5 of reference object C is determined as the identifier of target object A, that is, the predicted identifier 9 of target object A is corrected to identifier 5; where formula (1) is as follows:

[0120]

[0121] Where φ(m|μ, θ) represents the probabilistic similarity between the target object and the reference object, μ represents the pixel spectral amplitude of the target object, m represents the pixel spectral amplitude of the reference object, c represents the pixel spectral phase of the target object, θ represents the pixel spectral phase of the reference object, and σ 2 Indicates variance.

[0122] In one embodiment, for example, if the predicted identifier of a target object A is 9, and the difference between the pixel spectral amplitude of the target object A and the pixel spectral amplitude of a reference object B (which is also identified as 9) is less than or equal to a preset threshold, then the target object A and the reference object B are determined to be the same game character, i.e., the identifier of the target object A is 9. Since the predicted identifier 9 of the target object A is the same as the identifier 9, it is not necessary to correct the predicted identifier 9 of the target object A using a Gaussian mixture corrector.

[0123] Step A3: Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the predicted position information of each target object is corrected to determine the position information of each target object; the multiple reference objects include the reference object corresponding to the identifier.

[0124] Specifically, for example, based on the pixel spectral amplitude and pixel spectral phase of target object A (identified as 10), the pixel spectral amplitude and pixel spectral phase of the corresponding reference object B (identified as 10), the predicted position information of target object A is corrected to determine the position information of target object A; wherein, target object A and reference object B are the same game character.

[0125] In one embodiment, determining the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of multiple reference objects, and the pixel spectral phase of multiple reference objects includes:

[0126] By using Fourier transform, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the pixel spectrum of each target object, and the pixel spectrum amplitude and pixel spectrum phase of multiple reference objects are extracted from the pixel spectra of multiple reference objects.

[0127] Specifically, through Fourier transform, the pixel spectrum amplitude and phase of a target object are extracted from its pixel spectrum. This involves segmenting the pixel spectrum amplitude and phase to obtain the pixel spectrum amplitude and phase of the target object. Similarly, through Fourier transform, the pixel spectrum amplitude and phase of a reference object are extracted from its pixel spectrum. This involves segmenting the pixel spectrum amplitude and phase of the reference object to obtain the pixel spectrum amplitude and phase of the reference object.

[0128] In one embodiment, the predicted identifier of each target object is corrected based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of multiple reference objects, and the pixel spectral phase of multiple reference objects to determine the identifier of each target object, including:

[0129] Determine the difference between the pixel spectral amplitude of each target object and the pixel spectral amplitude of the reference object corresponding to the predicted label, where multiple reference objects include the reference object corresponding to the predicted label;

[0130] If the difference is greater than the preset threshold, the predicted label of each target object is corrected based on the pixel spectrum amplitude of each target object, the pixel spectrum phase of each target object, the pixel spectrum amplitude of other reference objects among multiple reference objects, and the pixel spectrum phase of other reference objects, and the label of each target object is determined; other reference objects do not include the reference object corresponding to the predicted label.

[0131] Specifically, for example, if the predicted identifier of a target object A is 9, and the difference between the pixel spectral amplitude of target object A and the pixel spectral amplitude of a reference object B (identified as 9) is greater than a preset threshold, then it is determined that target object A and reference object B are not the same game character. That is, the predicted identifier 9 of target object A is inaccurate and needs to be corrected. Based on the pixel spectral amplitude of target object A, the pixel spectral phase of target object A, the pixel spectral amplitudes of other reference objects besides reference object B, and the pixel spectral phases of other reference objects, the predicted identifier 9 of target object A is corrected to determine the identifier of target object A. For example, if the difference between the pixel spectrum amplitude of the target object A and the pixel spectrum amplitude of the reference object C (identified as 5) is less than or equal to a preset threshold, the difference between the pixel spectrum amplitude of the target object A and the pixel spectrum amplitude of the reference object D (identified as 6) is less than or equal to a preset threshold, and the difference between the pixel spectrum amplitude of the target object A and the pixel spectrum amplitude of the reference object E (identified as 7) is less than or equal to a preset threshold, then the probability similarity φ1 between the target object A and the reference object C, the probability similarity φ2 between the target object A and the reference object D, and the probability similarity φ3 between the target object A and the reference object E are calculated and determined by the Gaussian mixture corrector based on formula (1); the identifier of the reference object corresponding to the maximum probability similarity among φ1, φ2 and φ3 is determined as the identifier of the target object A. For example, if φ1 is the maximum probability similarity among φ1, φ2 and φ3, then the identifier 5 of the reference object C is determined as the identifier of the target object A, that is, the predicted identifier 9 of the target object A is corrected to identifier 5.

[0132] In one embodiment, the predicted identifier of each target object is corrected based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of other reference objects among multiple reference objects, and the pixel spectral phase of other reference objects, to determine the identifier of each target object, including:

[0133] Based on the pixel spectrum amplitude and pixel spectrum phase of each target object, as well as the pixel spectrum amplitude and pixel spectrum phase of other reference objects among multiple reference objects, the probabilistic similarity between each target object and each other reference object is determined, and the identifier of the reference object corresponding to the highest probability similarity among all probability similarities is determined as the identifier of each target object.

[0134] Specifically, for example, using a Gaussian mixture corrector, the probabilistic similarity φ1 between target object A and reference object C, the probabilistic similarity φ2 between target object A and reference object D, and the probabilistic similarity φ3 between target object A and reference object E are calculated and determined based on formula (1); the identifier of the reference object corresponding to the highest probabilistic similarity among φ1, φ2 and φ3 is determined as the identifier of target object A. For example, if φ2 is the highest probabilistic similarity among φ1, φ2 and φ3, then the identifier 6 of reference object D is determined as the identifier of target object A, that is, the predicted identifier 9 of target object A is corrected to identifier 6.

[0135] In one embodiment, the predicted position information of each target object is corrected based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude of the corresponding reference object, and the pixel spectral phase of the corresponding reference object, to determine the position information of each target object, including:

[0136] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the expected deviation value of the predicted position information of each target object is determined.

[0137] The location information of each target object is determined based on the predicted location information and the expected deviation value.

[0138] Specifically, for example, based on the pixel spectral amplitude and pixel spectral phase of target object A (identified as 10), the pixel spectral amplitude and pixel spectral phase of the corresponding reference object B (identified as 10), the expected deviation value of the predicted position information of target object A is determined; wherein, target object A and reference object B are the same game character, and the predicted position information of target object A can be the coordinates of the center position of target object A, that is, the coordinates of the pixel point at the center position of target object A, which includes the X coordinate and Y coordinate of the center position of target object A.

[0139] Based on the X-coordinate, Y-coordinate, and the expected deviation values ​​of the X-coordinate and Y-coordinate, the position information of target object A is determined. For example, the sum of the X-coordinate and the expected deviation value of the X-coordinate is calculated to obtain the X-coordinate in the position information of target object A; the sum of the Y-coordinate and the expected deviation value of the Y-coordinate is calculated to obtain the Y-coordinate in the position information of target object A.

[0140] Applying the embodiments of this application has at least the following beneficial effects:

[0141] Target separation is performed on the target image in frame t to obtain the pixel spectrum and predicted location information of each target object in frame t; ID labeling is performed on each target object in frame t, that is, the ID (predicted identifier) ​​of each target object in frame t is predicted; ID correction and predicted location information correction are performed on each target object in frame t; thereby improving the detection success rate of each target object in the target image, that is, improving the detection success rate of multiple target objects in the target image.

[0142] To better understand the methods provided in the embodiments of this application, the solutions of the embodiments of this application will be further explained below with reference to specific application scenarios.

[0143] In a specific application scenario, such as a scenario involving the detection of multiple game characters in a game, see [link to example]. Figure 3 This illustrates the processing flow of a target detection method, such as... Figure 3 As shown, the processing flow of the target detection method provided in this application embodiment includes the following steps:

[0144] S301, the detector performs target separation on the target image of the game's t-th frame.

[0145] Specifically, the detector can be located on a server or on a terminal. The target image is the image at frame t in the game, and the reference image is the image at frame t-1 in the game. The target image contains multiple game characters, and the reference image also contains multiple game characters. For example, such as... Figure 4 As shown, target separation is performed on the target image of the game's t-th frame. This target separation includes steps B1-B5:

[0146] Step B1: Convolve the target image with each of the 12 convolutional kernels in the (3×3, 12) convolutional layer to obtain the sub-vector features corresponding to each of the 12 convolutional kernels; concatenate the sub-vector features corresponding to the 12 convolutional kernels to obtain the feature vector of the target image. The feature vector of the target image can be a clustered convolutional matrix.

[0147] Step B2: The feature vector of the target image is reduced in dimensionality by a max pooling layer to obtain the reduced feature vector.

[0148] Step B3 involves convolving the dimensionality-reduced feature vectors with a 3×3 convolution kernel that has a smoothing filter, and then performing a Fourier transform to obtain the pixel spectrum of each target object in the target image.

[0149] Step B4: Normalize and upsample the pixel spectrum of each target object to obtain the processed pixel spectrum of each target object.

[0150] Step B5: The processed pixel spectrum of each target object and the pixels in the target image are input into the pixel-level classifier. The pixel-level classifier classifies the pixels in the target image based on the processed pixel spectrum of each target object, and determines the predicted position information of each target object, that is, the coordinates of the center position pixel of each target object. This achieves target separation in the target image of frame t, distinguishing the target objects and the background in the target image, that is, distinguishing the game characters and the game background in the game.

[0151] S302, the detector marks each target object and obtains the predicted label of each target object.

[0152] Specifically, the target object is a game character in the game. For example, such as Figure 4 As shown, each target object is marked. For example, based on the amplitude and phase of the center pixel B of reference object A in the reference image of frame t-1, the amplitude and phase of the center pixel B in the target image of frame t are determined. Based on the amplitude, phase, and magnitude of the center pixel B in frame t-1, the movement deviation of the center pixel B between frame t-1 and frame t is determined, obtaining the trajectory of the center pixel B moving from position M in frame t-1 to position N in frame t. This allows the prediction of the corresponding position of the center pixel B of reference object A in the target image. If the center pixel of a target object is at the corresponding position, the predicted identifier of reference object A can be used. For example, if the identifier of reference object A is 5, the predicted identifier of the target object at the corresponding position can be 5, meaning that the target object and reference object A are predicted to be the same game character.

[0153] It should be noted that similar and adjacent target objects are prone to mislabeling, and the predicted position information of the target object may be deviated due to noise. Therefore, it is necessary to correct the predicted label and the predicted position information of the target object through step S304.

[0154] S303, the detector stores the processed pixel spectrum of each target object and the predicted position information of each target object into the detection bucket corresponding to each target object.

[0155] Specifically, for example, such as Figure 4 As shown, the processed pixel spectrum and predicted location information of each target object are stored in the corresponding detection bucket for each target object. The detection bucket can be a table, and each target object's detection bucket stores the processed pixel spectrum and predicted location information for each target object.

[0156] S304, the detector corrects the predicted identifier and predicted position information of each target object through a Gaussian mixture corrector, and determines the identifier and position information of each target object.

[0157] Specifically, for example, such as Figure 4 As shown, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the processed pixel spectrum of each target object through Fourier transform, and the pixel spectrum amplitude and pixel spectrum phase of multiple reference objects are extracted from the pixel spectra of multiple reference objects; the predicted identifier and predicted position information of each target object are corrected through a Gaussian mixture corrector; wherein, the correction of the predicted identifier and predicted position information of each target object through a Gaussian mixture corrector includes steps C1-C2:

[0158] Step C1: Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of multiple reference objects, the predicted label of each target object is corrected by a Gaussian mixture corrector to determine the label of each target object.

[0159] Step C2: Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, a Gaussian mixture corrector is used to correct the predicted position information of each target object to determine the position information of each target object; the multiple reference objects include the reference object corresponding to the identifier.

[0160] The pixel spectrum and position information of the reference object are stored in the detection bucket of the reference object, while the processed pixel spectrum and predicted position information of the target object are stored in the detection bucket of the target object.

[0161] S305, the terminal displays each target object and its identifier at the location corresponding to the location information of each target object.

[0162] Specifically, for example, such as Figure 5As shown, in a MOBA game, target objects (game characters) to be detected are set. When the MOBA game starts, the detector can actively identify multiple moving target objects and treat them as detection objects. The detector automatically marks multiple target objects with detection boxes of different colors to distinguish different target objects; for example, target object A is marked with a red detection box, target object B is marked with a blue detection box, and target object C is marked with a yellow detection box. During the target object detection process, the detector will continuously record and detect different target objects; if a target object disappears in the game, the detection box of that target object will also disappear.

[0163] Applying the embodiments of this application has at least the following beneficial effects:

[0164] Target segmentation is performed on the target image of frame t in the game to obtain the pixel spectrum and predicted position information of each target object (game character) in frame t; ID tagging is performed on each target object in frame t, that is, the ID (predicted identifier) ​​of each target object in frame t is predicted; ID correction and predicted position information correction are performed on each target object in frame t; thereby improving the detection success rate of each target object in the target image, that is, improving the detection success rate of multiple target objects in the target image.

[0165] This application also provides a target detection device, the structural schematic diagram of which is shown below. Figure 6 As shown, the target detection device 60 includes a first processing module 601, a second processing module 602, a third processing module 603, and a fourth processing module 604.

[0166] The first processing module 601 is used to acquire a target image and a reference image, wherein the target image is the next frame image adjacent to the reference image;

[0167] The second processing module 602 is used to determine the pixel spectrum of each target object in the target image and the predicted position information of each target object.

[0168] The third processing module 603 is used to determine the predicted identifier of each target object based on the reference image;

[0169] The fourth processing module 604 is used to correct the predicted identifier and predicted position information of each target object based on the pixel spectrum of each target object, the predicted position information of each target object, the pixel spectrum of multiple reference objects in the reference image, and the position information of multiple reference objects, and to determine the identifier and position information of each target object.

[0170] In one embodiment, the second processing module 602 is specifically used for:

[0171] Determine the feature vector of the target image;

[0172] Based on the feature vector of the target image, determine the pixel spectrum of each target object in the target image;

[0173] Based on the pixel spectrum of each target object, the pixels in the target image are classified by a preset pixel-level classifier to determine the predicted position information of each target object. The predicted position information is the coordinates of the center pixel of each target object.

[0174] In one embodiment, the second processing module 602 is specifically used for:

[0175] The target image is convolved with each convolution kernel in the preset convolutional layer to obtain the sub-vector features corresponding to each convolution kernel. The convolutional layer includes multiple convolution kernels.

[0176] The feature vectors of the target image are obtained by concatenating the features of each sub-vector.

[0177] In one embodiment, the second processing module 602 is specifically used for:

[0178] The feature vector of the target image is reduced in dimension to obtain the reduced feature vector.

[0179] The reduced feature vectors are then smoothed and subjected to Fourier transform to obtain the pixel spectrum of each target object in the target image.

[0180] In one embodiment, the third processing module 603 is specifically used for:

[0181] Based on the amplitude and phase of the center pixel of any reference object in the reference image, determine the corresponding position of the center pixel of any reference object in the target image;

[0182] If the center pixel of each target object is in the corresponding position, then the identifier of any reference object is determined as the predicted identifier of each target object.

[0183] In one embodiment, the fourth processing module 604 is specifically used for:

[0184] Determine the pixel spectral amplitude and pixel spectral phase of each target object, as well as the pixel spectral amplitude and pixel spectral phase of multiple reference objects;

[0185] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of multiple reference objects, the predicted label of each target object is corrected to determine the label of each target object.

[0186] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the predicted position information of each target object is corrected to determine the position information of each target object; multiple reference objects include the reference object corresponding to the identifier.

[0187] In one embodiment, the fourth processing module 604 is specifically used for:

[0188] By using Fourier transform, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the pixel spectrum of each target object, and the pixel spectrum amplitude and pixel spectrum phase of multiple reference objects are extracted from the pixel spectra of multiple reference objects.

[0189] In one embodiment, the fourth processing module 604 is specifically used for:

[0190] Determine the difference between the pixel spectral amplitude of each target object and the pixel spectral amplitude of the reference object corresponding to the predicted label, where multiple reference objects include the reference object corresponding to the predicted label;

[0191] If the difference is greater than the preset threshold, the predicted label of each target object is corrected based on the pixel spectrum amplitude of each target object, the pixel spectrum phase of each target object, the pixel spectrum amplitude of other reference objects among multiple reference objects, and the pixel spectrum phase of other reference objects, and the label of each target object is determined; other reference objects do not include the reference object corresponding to the predicted label.

[0192] In one embodiment, the fourth processing module 604 is specifically used for:

[0193] Based on the pixel spectrum amplitude and pixel spectrum phase of each target object, as well as the pixel spectrum amplitude and pixel spectrum phase of other reference objects among multiple reference objects, the probabilistic similarity between each target object and each other reference object is determined, and the identifier of the reference object corresponding to the highest probability similarity among all probability similarities is determined as the identifier of each target object.

[0194] In one embodiment, the fourth processing module 604 is specifically used for:

[0195] Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the corresponding reference object, the expected deviation value of the predicted position information of each target object is determined.

[0196] The location information of each target object is determined based on the predicted location information and the expected deviation value.

[0197] Applying the embodiments of this application has at least the following beneficial effects:

[0198] The process involves acquiring a target image and a reference image. The target image is the next frame adjacent to the reference image, i.e., the target image is the image of frame t, and the reference image is the image of frame t-1. The process then involves determining the pixel spectrum and predicted location information of each target object in the target image. Based on the reference image, a predicted identifier for each target object is determined. Based on the pixel spectrum and predicted location information of each target object, as well as the pixel spectrum and location information of multiple reference objects in the reference image, the predicted identifier and predicted location information of each target object are corrected, and the identifier and location information of each target object are determined. This process is repeated to separate the target image in frame t, obtaining the pixel spectrum and predicted location information (predicted location information) of each target object in frame t. Each target object in frame t is then assigned an ID, i.e., the ID (predicted identifier) ​​of each target object in frame t is predicted. Finally, ID correction and predicted location information correction are performed for each target object in frame t. This improves the detection success rate of each target object in the target image, thus improving the detection success rate of multiple target objects in the target image.

[0199] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 7 As shown, Figure 7 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0200] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0201] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0202] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0203] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0204] Electronic devices include, but are not limited to, terminals and servers.

[0205] Applying the embodiments of this application has at least the following beneficial effects:

[0206] The process involves acquiring a target image and a reference image. The target image is the next frame adjacent to the reference image, i.e., the target image is the image of frame t, and the reference image is the image of frame t-1. The process then involves determining the pixel spectrum and predicted location information of each target object in the target image. Based on the reference image, a predicted identifier for each target object is determined. Based on the pixel spectrum and predicted location information of each target object, as well as the pixel spectrum and location information of multiple reference objects in the reference image, the predicted identifier and predicted location information of each target object are corrected, and the identifier and location information of each target object are determined. This process is repeated to separate the target image in frame t, obtaining the pixel spectrum and predicted location information (predicted location information) of each target object in frame t. Each target object in frame t is then assigned an ID, i.e., the ID (predicted identifier) ​​of each target object in frame t is predicted. Finally, ID correction and predicted location information correction are performed for each target object in frame t. This improves the detection success rate of each target object in the target image, thus improving the detection success rate of multiple target objects in the target image.

[0207] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0208] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0209] Based on the same principles as the methods provided in the embodiments of this application, the embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in any of the optional embodiments of this application described above.

[0210] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0211] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A target detection method, characterized in that, include: Acquire a target image and a reference image, wherein the target image is the next frame image adjacent to the reference image, the target image includes multiple target objects, and the reference image includes multiple reference objects; Determine the pixel spectrum of each target object in the target image and the predicted location information of each target object; Based on the reference image, a predicted identifier for each target object is determined; Determine the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects; Based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects, the predicted identifier of each target object is corrected to determine the identifier of each target object; Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the reference object corresponding to the identifier, the predicted position information of each target object is corrected to determine the position information of each target object; the plurality of reference objects include the reference object corresponding to the identifier.

2. The method according to claim 1, characterized in that, Determining the pixel spectrum of each target object in the target image and the predicted location information of each target object includes: Determine the feature vector of the target image; Based on the feature vector of the target image, determine the pixel spectrum of each target object in the target image; Based on the pixel spectrum of each target object, the pixels in the target image are classified by a preset pixel-level classifier to determine the predicted position information of each target object, wherein the predicted position information is the coordinates of the center pixel of each target object.

3. The method according to claim 2, characterized in that, Determining the feature vector of the target image includes: The target image is convolved with each convolution kernel in a preset convolutional layer to obtain the sub-vector features corresponding to each convolution kernel. The convolutional layer includes multiple convolution kernels. The feature vectors of the target image are obtained by concatenating the features of each sub-vector.

4. The method according to claim 2, characterized in that, Determining the pixel spectrum of each target object in the target image based on the feature vector of the target image includes: The feature vector of the target image is reduced in dimension to obtain the reduced feature vector. The reduced feature vectors are then subjected to smoothing filtering and Fourier transform to obtain the pixel spectrum of each target object in the target image.

5. The method according to claim 1, characterized in that, Determining the predicted identifier of each target object based on the reference image includes: Based on the amplitude and phase of the center pixel of any reference object in the reference image, the corresponding position of the center pixel of any reference object in the target image is determined. If the center pixel of each target object is located at the corresponding position, then the identifier of any reference object is determined as the predicted identifier of each target object.

6. The method according to claim 1, characterized in that, Determining the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects includes: By using Fourier transform, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the pixel spectrum of each target object, and the pixel spectrum amplitude and pixel spectrum phase of the multiple reference objects are extracted from the pixel spectrum of the multiple reference objects.

7. The method according to claim 1, characterized in that, The step of correcting the predicted identifier of each target object based on the pixel spectral amplitude, pixel spectral phase, pixel spectral amplitude, and pixel spectral phase of each target object, and determining the identifier of each target object, includes: Determine the difference between the pixel spectral amplitude of each target object and the pixel spectral amplitude of the reference object corresponding to the prediction identifier, wherein the plurality of reference objects include the reference object corresponding to the prediction identifier; If the difference is greater than a preset threshold, the predicted identifier of each target object is corrected based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of other reference objects among the plurality of reference objects, and the pixel spectral phase of other reference objects, and the identifier of each target object is determined; the other reference objects do not include the reference object corresponding to the predicted identifier.

8. The method according to claim 7, characterized in that, The step of correcting the predicted identifier of each target object based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of other reference objects among the plurality of reference objects, and the pixel spectral phase of the other reference objects, and determining the identifier of each target object, includes: Based on the pixel spectrum amplitude and pixel spectrum phase of each target object, the pixel spectrum amplitude and pixel spectrum phase of other reference objects among the plurality of reference objects, the probability similarity between each target object and each of the other reference objects is determined, and the identifier of the reference object corresponding to the highest probability similarity among the probability similarities is determined as the identifier of each target object.

9. The method according to claim 1, characterized in that, The step of correcting the predicted position information of each target object based on the pixel spectral amplitude, pixel spectral phase, pixel spectral amplitude of the reference object corresponding to the identifier, and pixel spectral phase of the reference object corresponding to the identifier, and determining the position information of each target object, includes: Based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the reference object corresponding to the identifier, and the pixel spectral phase of the reference object corresponding to the identifier, the expected deviation value of the predicted position information of each target object is determined; Based on the predicted location information and the expected deviation value, the location information of each target object is determined.

10. A target detection device, characterized in that, include: The first processing module is used to acquire a target image and a reference image, wherein the target image is the next frame image adjacent to the reference image, the target image includes multiple target objects, and the reference image includes multiple reference objects; The second processing module is used to determine the pixel spectrum of each target object in the target image and the predicted location information of each target object; The third processing module is used to determine the predicted identifier of each target object based on the reference image; The fourth processing module is used to determine the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects; Based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects, the predicted identifier of each target object is corrected to determine the identifier of each target object; Based on the pixel spectral amplitude and pixel spectral phase of each target object, the pixel spectral amplitude and pixel spectral phase of the reference object corresponding to the identifier, the predicted position information of each target object is corrected to determine the position information of each target object; the plurality of reference objects include the reference object corresponding to the identifier.

11. The apparatus according to claim 10, characterized in that, When determining the pixel spectrum of each target object in the target image and the predicted location information of each target object, the second processing module is specifically used for: Determine the feature vector of the target image; Based on the feature vector of the target image, determine the pixel spectrum of each target object in the target image; Based on the pixel spectrum of each target object, the pixels in the target image are classified by a preset pixel-level classifier to determine the predicted position information of each target object, wherein the predicted position information is the coordinates of the center pixel of each target object.

12. The apparatus according to claim 11, characterized in that, When determining the feature vector of the target image, the second processing module is specifically used for: The target image is convolved with each convolution kernel in a preset convolutional layer to obtain the sub-vector features corresponding to each convolution kernel. The convolutional layer includes multiple convolution kernels. The feature vectors of the target image are obtained by concatenating the features of each sub-vector.

13. The apparatus according to claim 11, characterized in that, When determining the pixel spectrum of each target object in the target image based on the feature vector of the target image, the second processing module is specifically used for: The feature vector of the target image is reduced in dimension to obtain the reduced feature vector. The reduced feature vectors are then subjected to smoothing filtering and Fourier transform to obtain the pixel spectrum of each target object in the target image.

14. The apparatus according to claim 10, characterized in that, When determining the predicted identifier of each target object based on the reference image, the third processing module is specifically used for: Based on the amplitude and phase of the center pixel of any reference object in the reference image, the corresponding position of the center pixel of any reference object in the target image is determined. If the center pixel of each target object is located at the corresponding position, then the identifier of any reference object is determined as the predicted identifier of each target object.

15. The apparatus according to claim 10, characterized in that, The fourth processing module, when determining the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the plurality of reference objects, and the pixel spectral phase of the plurality of reference objects, is specifically used for: By using Fourier transform, the pixel spectrum amplitude and pixel spectrum phase of each target object are extracted from the pixel spectrum of each target object, and the pixel spectrum amplitude and pixel spectrum phase of the multiple reference objects are extracted from the pixel spectrum of the multiple reference objects.

16. The apparatus according to claim 10, characterized in that, When the fourth processing module corrects the predicted identifier of each target object based on the pixel spectral amplitude, pixel spectral phase, pixel spectral amplitude, and pixel spectral phase of each target object, and determines the identifier of each target object, it is specifically used for: Determine the difference between the pixel spectral amplitude of each target object and the pixel spectral amplitude of the reference object corresponding to the prediction identifier, wherein the plurality of reference objects include the reference object corresponding to the prediction identifier; If the difference is greater than a preset threshold, the predicted identifier of each target object is corrected based on the pixel spectrum amplitude of each target object, the pixel spectrum phase of each target object, the pixel spectrum amplitude of other reference objects among the plurality of reference objects, and the pixel spectrum phase of other reference objects, and the identifier of each target object is determined. The other reference objects do not include the reference object corresponding to the prediction identifier.

17. The apparatus according to claim 16, characterized in that, The fourth processing module, when correcting the predicted identifier of each target object based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of other reference objects among the plurality of reference objects, and the pixel spectral phase of the other reference objects, and determining the identifier of each target object, is specifically used for: Based on the pixel spectrum amplitude and pixel spectrum phase of each target object, the pixel spectrum amplitude and pixel spectrum phase of other reference objects among the plurality of reference objects, the probability similarity between each target object and each of the other reference objects is determined, and the identifier of the reference object corresponding to the highest probability similarity among the probability similarities is determined as the identifier of each target object.

18. The apparatus according to claim 10, characterized in that, When the fourth processing module corrects the predicted position information of each target object based on the pixel spectral amplitude, pixel spectral phase, pixel spectral amplitude of the reference object corresponding to the identifier, and pixel spectral phase of the reference object corresponding to the identifier, and determines the position information of each target object, it is specifically used for: Based on the pixel spectral amplitude of each target object, the pixel spectral phase of each target object, the pixel spectral amplitude of the reference object corresponding to the identifier, and the pixel spectral phase of the reference object corresponding to the identifier, the expected deviation value of the predicted position information of each target object is determined; Based on the predicted location information and the expected deviation value, the location information of each target object is determined.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-9.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-9.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-9.

Images