Player identification method, device and equipment

Abstract

The invention discloses a player recognition method, device and equipment, and the method comprises the steps: respectively obtaining three-dimensional human body grids of players and interference persons from a video image sequence of two confrontation parties with the clothes colors meeting the requirements of a motion scene; extracting a vertex color value from the video image according to a vertex visibility state and a grid projection relationship, and mapping the vertex color value to a UV map with a fixed resolution through a predefined vertex index to generate a human body texture representation with an unchanged posture; and then classifying the UV chartlet through a pre-trained classification model to obtain an identification result of confrontation two-party players in the video image. The classification model is trained according to the clothing features of the players of the two parties, so that background interference (including referees and audiences) can be accurately eliminated; moreover, the local features of the clothes are mapped to the fixed positions through the UV chartlet, the fine extraction of the clothes of the human body can be realized, the interference of the posture of the human body is eliminated, and the recognition accuracy is improved.

Description

Technical Field

[0001] This specification relates to the field of computer vision technology, and more particularly to a contestant recognition method, apparatus, and device. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence and computer vision technologies, intelligent analysis of sports videos has become an important application area in the field of sports technology. By automatically identifying and analyzing elements such as people, actions, and scenes in sports videos, technical teams can provide coaches, athletes, and viewers with deeper data support. These technologies are widely used in various fields such as event broadcasting, tactical analysis, athlete performance evaluation, and injury prevention, improving the fairness and entertainment value of competitions while also promoting the intelligentization of athlete training.

[0003] Taking boxing as an example, identifying the red and blue boxers is a fundamental step in subsequent intelligent analysis of techniques and tactics. Unlike sports like football and tennis, boxing matches take place in a small arena, close to the audience, and the referee is often in the same ring as the boxer. It's difficult to eliminate background figures simply by adjusting the camera angle. Using multi-object tracking algorithms to analyze match videos would involve tracking all individuals. Therefore, an algorithm is needed to identify the competing fighters from the background figures in a two-on-two combat scenario. Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide a method, apparatus, and device for identifying athletes, so as to improve the accuracy of athlete identification in a two-on-two competitive sports scenario.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a contestant identification method is proposed, comprising:

[0007] Acquire a video image sequence containing a motion scene of opposing sides, wherein the clothing colors of the opposing sides conform to the regulations of the motion scene;

[0008] The three-dimensional human body mesh of each contestant and the interfering person is obtained from the image frames in the sequence. The three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices.

[0009] Based on the vertex visibility state and mesh projection relationship, vertex color values ​​are extracted from the video image;

[0010] The color values ​​of each mesh vertex are mapped to a UV map of fixed resolution using a predefined vertex index to generate a pose-invariant human texture representation.

[0011] The UV map is input into a pre-trained classification model to obtain a classification result, wherein the classification model is trained based on the UV map dataset in the motion scene;

[0012] The opposing players in the image frame are identified based on the classification results.

[0013] In some embodiments, obtaining the three-dimensional human body meshes of each contestant and interfering person from the image frames in the sequence includes:

[0014] For the image frame, the human detection bounding boxes and segmentation masks of each contestant and interfering person are obtained according to the human detection algorithm or tracking algorithm.

[0015] The human body region image is cropped according to the human body detection box, and the human body region image is processed by a pre-trained human pose estimation model to obtain the vertex 3D coordinates and camera parameters.

[0016] The three-dimensional coordinates of the vertices are projected onto a two-dimensional image plane using the camera parameters.

[0017] Vertex visibility status is determined based on projection coordinates and segmentation mask.

[0018] In some embodiments, determining the vertex visibility state based on the projection coordinates and the segmentation mask includes:

[0019] For each vertex, if the projected coordinates are located in the region with a segmentation mask of 1 and are not occluded by other targets, it is marked as visible.

[0020] Otherwise, mark it as invisible.

[0021] In some embodiments, extracting vertex color values ​​from the video image based on the vertex visibility state and mesh projection relationship includes:

[0022] For vertices whose visibility is visible, extract the color value of the corresponding position in the two-dimensional projection coordinates from the video image;

[0023] For vertices whose visibility is not visible, set the color value to zero, or determine the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame's time window.

[0024] In some embodiments, determining the vertex color value based on the vertex's color value in the visible state in other frames within the current frame setting time window includes:

[0025] In real-time processing, the vertex color value is determined based on the color value of the vertex in the visible state most recently before the current frame.

[0026] In offline processing, the vertex color value is determined based on the color value of the vertex in its most recent visible state after the current frame; or

[0027] The vertex color value is determined based on the average color value of all visible vertices that appear within a set time window.

[0028] In some embodiments, mapping the coordinates and color values ​​of each mesh vertex to a fixed-resolution UV map includes:

[0029] The vertices of the 3D mesh are mapped to the UV plane using a predefined mapping function, wherein the UV plane is discretized into a rasterized texture.

[0030] For each grid position, obtain at least one vertex mapped to that position;

[0031] The grid is filled according to the color value corresponding to the at least one vertex.

[0032] In some embodiments, filling the raster according to the color value corresponding to the at least one vertex includes at least one of the following:

[0033] Based on the distance weights between the two-dimensional projection position of each vertex and the center of the grid, a blended color value is generated to fill the grid.

[0034] The grid is filled with the color value of the vertex whose two-dimensional projection position is closest to the grid center.

[0035] In some embodiments, the method further includes:

[0036] Construct a UV map dataset that includes three categories: opposing sides and interfering personnel;

[0037] The classification model is trained using the UV mapping dataset.

[0038] According to a second aspect of one or more embodiments of this specification, a contestant identification device is provided, comprising:

[0039] The first acquisition unit is used to acquire a video image sequence containing a motion scene of opposing sides, wherein the clothing colors of the opposing sides conform to the regulations of the motion scene;

[0040] The second acquisition unit is used to acquire the three-dimensional human body mesh of each contestant and the interfering person from the image frames in the sequence, and the three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices.

[0041] An extraction unit is used to extract vertex color values ​​from the video image based on the vertex visibility state and the mesh projection relationship;

[0042] The mapping unit is used to map the color values ​​of each mesh vertex to a UV map of fixed resolution through a predefined vertex index, thereby generating a pose-invariant human texture representation.

[0043] A classification unit is used to input the UV map into a pre-trained classification model to obtain a classification result, wherein the classification model is trained based on the UV map dataset in the motion scene;

[0044] The identification unit is used to identify the two opposing players in the image frame based on the classification result.

[0045] In some embodiments, the second acquisition unit is specifically used for:

[0046] For the image frame, the human detection bounding boxes and segmentation masks of each contestant and interfering person are obtained according to the human detection algorithm or tracking algorithm.

[0047] The human body region image is cropped according to the human body detection box, and the human body region image is processed by a pre-trained human pose estimation model to obtain the vertex 3D coordinates and camera parameters.

[0048] The three-dimensional coordinates of the vertices are projected onto a two-dimensional image plane using the camera parameters.

[0049] Vertex visibility status is determined based on projection coordinates and segmentation mask.

[0050] In some embodiments, when the second acquisition unit is used to determine the vertex visibility state based on the projection coordinates and the segmentation mask, it is specifically used for:

[0051] For each vertex, if the projected coordinates are located in the region with a segmentation mask of 1 and are not occluded by other targets, it is marked as visible.

[0052] Otherwise, mark it as invisible.

[0053] In some embodiments, the extraction unit is specifically used for:

[0054] For vertices whose visibility is visible, extract the color value of the corresponding position of the two-dimensional projection coordinates from the video image;

[0055] For vertices whose visibility is not visible, set the color value to zero, or determine the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame's time window.

[0056] In some embodiments, when the extraction unit determines the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame time window, it is specifically used for:

[0057] In real-time processing, the vertex color value is determined based on the color value of the vertex in the visible state most recently before the current frame.

[0058] In offline processing, the vertex color value is determined based on the color value of the vertex in its most recent visible state after the current frame; or

[0059] The vertex color value is determined based on the average color value of all visible vertices that appear within a set time window.

[0060] In some embodiments, the mapping unit is specifically used to: map vertices of a 3D mesh to a UV plane using a predefined mapping function, wherein the UV plane is discretized into a rasterized texture; for each raster position, obtain at least one vertex mapped to that position; and fill the raster according to the color value corresponding to the at least one vertex.

[0061] In some embodiments, when the mapping unit is used to fill the grid according to the color value corresponding to the at least one vertex, it is specifically used to: generate a grid filled with fused color values ​​according to the distance weight between the two-dimensional projection position of each vertex and the grid center; and fill the grid according to the color value of the vertex whose two-dimensional projection position is closest to the grid center.

[0062] In some embodiments, the apparatus further includes a training unit for: constructing a UV map dataset containing three categories: adversaries and interfering personnel; and training the classification model using the UV map dataset.

[0063] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising:

[0064] processor;

[0065] Memory used to store processor-executable instructions;

[0066] The processor implements the steps of the method proposed in the above embodiments by running the executable instructions.

[0067] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method proposed in the above embodiments.

[0068] According to a fifth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method proposed in the above embodiments.

[0069] The contestant identification method proposed in this specification involves obtaining 3D human body meshes for each contestant and interfering personnel from a video image sequence containing opposing sides whose clothing colors conform to the sports scene specifications. Based on vertex visibility states and mesh projection relationships, vertex color values ​​are extracted from the video images and mapped to a fixed-resolution UV map using predefined vertex indices, generating a pose-invariant human texture representation. Then, a pre-trained classification model is used to classify the UV map to obtain the identification results of the opposing contestants in the video images. Since this classification model is trained on the clothing features of both contestants, it can accurately eliminate background interference (including referees and spectators); and the UV map maps local clothing features to fixed positions, enabling fine extraction of human clothing and eliminating interference from human posture, thus improving recognition accuracy. Attached Figure Description

[0070] Figure 1 This is a flowchart of a contestant identification method provided in an exemplary embodiment.

[0071] Figure 2 This is a flowchart of a method for obtaining a three-dimensional human body mesh, provided in an exemplary embodiment.

[0072] Figure 3 This is a schematic diagram of a UV mapping sample provided in an exemplary embodiment.

[0073] Figure 4 This is a block diagram of a contestant identification device provided in an exemplary embodiment.

[0074] Figure 5 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Detailed Implementation

[0075] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0076] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0077] Figure 1 A flowchart of a contestant identification method provided in an embodiment of this application is shown. The method includes:

[0078] Step 101: Obtain a video image sequence containing the motion scene of both opposing sides.

[0079] A sport involving two opposing sides refers to a competitive sport in which two athletes engage in direct physical confrontation within a confined space and are distinguished by the color of their clothing, such as boxing, wrestling, and judo. In these sports, the colors of the athletes' clothing must conform to the specific requirements of the sport; for example, in boxing, one athlete must wear red and the other blue.

[0080] The video image sequence can be a continuous sequence of images captured by the camera, or it can be a real-time video stream.

[0081] Step 102: Obtain the three-dimensional human body mesh of each contestant and the interfering person from the image frames in the sequence. The three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices.

[0082] For each frame in a video image sequence, object detection models (such as Mask-RCNN) or tracking algorithms (DeepSORT algorithm) can be used to obtain the human bounding boxes and segmentation masks for each contestant and interfering person, thereby determining the 3D human mesh of each detected contestant and interfering person. A 3D human mesh is a digital human surface model used to represent the body shape and position of the human body in three-dimensional space. Its core elements include vertices and topology, and vertices, in addition to having 3D coordinates, also have visibility state information. Visibility state is a binary label indicating whether each vertex is observable in the current frame image. In the visible state, the body part corresponding to the vertex is not occluded and is within the image range, while in the invisible state, the vertex cannot be observed due to occlusion or being outside the field of view.

[0083] Step 103: Extract vertex color values ​​from the video image based on the vertex visibility state and mesh projection relationship.

[0084] Mesh projection relationship refers to the geometric mapping relationship between the vertices of a 3D human body mesh and the 2D video image plane, which is transformed through spatial coordinates established by the camera model. For vertices with visible visibility, the color value of the corresponding position of the 2D projection coordinates is extracted from the video image; for vertices with invisible visibility, the color value is set to zero, or the vertex color value is determined based on the color value of the vertex in the visible state in other frames within the current frame's time window.

[0085] Step 104: Map the color values ​​of each mesh vertex to a UV map of fixed resolution using a predefined vertex index to generate a pose-invariant human texture representation.

[0086] Predefined vertex index mapping refers to establishing a fixed mapping relationship between the human body topology and the texture plane by binding each vertex ID of the 3D mesh to fixed UV coordinates. When the human body pose changes, the coordinates of the 3D mesh vertices also change accordingly. However, the UV map, through the fixed mapping of vertex indices, is independent of the human body pose. It can eliminate the texture position changes caused by rotation, translation, scaling, etc., and the same body area (such as the chest) will be in the same position in the UV map of different players.

[0087] Furthermore, it can still clearly present texture details at a fixed resolution. This means it achieves the conversion of dynamic human figures into static texture representations, laying the foundation for subsequent contestant recognition.

[0088] For example, a UV mapping function can be constructed to map the vertices of a human body mesh onto a fixed-size UV ​​map. Assuming the UV map size is D×D, the UV mapping function is u,v=f(j), u,v∈[1,D], thus mapping the color information of the human body mesh onto the UV map. For visible vertices on the human body mesh, the corresponding position on the UV map represents the color of that vertex in the video frame. For invisible vertices, the RGB value of the corresponding position on the UV map is set to 0. If an adjustment value has been set for an invisible vertex, the corresponding RGB value on the UV map is set to that adjustment value, ultimately resulting in the UV map.

[0089] Step 105: Input the UV map into the pre-trained classification model to obtain the classification result.

[0090] The classification model is trained on a UV map dataset specific to this sporting scene. This dataset provides standardized UV map samples of both the competing player and the interfering player. Taking a boxing match as an example, the model provides samples for three categories: the red corner player, the blue corner player, and the interfering player. Figure 3 As shown.

[0091] In one example, the structure of the UV mapping classification model can be as follows: First, a ResNet-18 model is used to extract visual features. Then, a linear layer is used to convert the multidimensional output of the ResNet-18 into three dimensions to achieve three-dimensional classification. Those skilled in the art should understand that the above structure is merely an example, and other classification model structures can be used; this disclosure is not intended to limit the scope of the classification model.

[0092] Step 106: Identify the opposing players in the image frame based on the classification results.

[0093] In this embodiment, three-dimensional human body meshes of each competitor and interfering personnel are obtained from a video image sequence containing opposing sides whose clothing colors conform to the sports scene specifications. Vertex color values ​​are extracted from the video images based on vertex visibility states and mesh projection relationships, and mapped to a fixed-resolution UV map using predefined vertex indices to generate a pose-invariant human texture representation. Then, a pre-trained classification model is used to classify the UV map to obtain the identification results of the opposing competitors in the video images. Since this classification model is trained on the clothing features of both competitors, it can accurately eliminate background interference (including referees and spectators). Furthermore, the UV map maps local clothing features to fixed positions, enabling fine extraction of human clothing and eliminating interference from human posture, thus improving recognition accuracy.

[0094] In some embodiments, such as Figure 2 The method shown obtains the 3D human body mesh of each contestant and interfering personnel. The method includes:

[0095] Step 201: For the image frame, obtain the human detection bounding box (bbox) and segmentation mask for each contestant and interfering person according to the human detection algorithm or tracking algorithm.

[0096] Specifically, for the input video sequence, an architecture of human detection initialization, tracking maintenance, and periodic correction can be adopted. That is, the initial frame can be generated using a human detection algorithm, and then maintained using a tracking algorithm. If the tracking conditions are not met, the human detection algorithm is re-executed, thereby optimizing computational efficiency while ensuring accuracy.

[0097] By performing a human detection algorithm or tracking algorithm on a single image frame in a video image sequence, it is possible to detect human bounding boxes and extract human segmentation masks, as shown in the following formula.

[0098]

[0099] Where F represents the human detection model, V iLet i represent the i-th frame of the match video, and K represent the number of people detected in the frame.

[0100] Step 202: Cropping the human body region image according to the human body detection box, and processing the human body region image through a pre-trained human pose estimation model to obtain the vertex 3D coordinates and camera parameters.

[0101] Based on the detected bounding boxes, each detected human body region is cropped out, and a human pose estimation model (such as DensePose-RCNN) is used to estimate the human body mesh, as shown in the following formula.

[0102]

[0103] Where H represents the human pose estimation model, and bbox k This represents the bounding box (bbox) of the detected k-th person, crop represents the image cropping operation based on the bbox, and verts 3D This represents the estimated 3D coordinates of the human body mesh vertices, and cam represents the camera parameters for that person, including the translation parameter t and the scaling parameter s.

[0104] Step 203: Project the three-dimensional coordinates of the vertex onto the two-dimensional image plane using the camera parameters.

[0105] By using camera parameters and the 3D coordinates of the human body mesh vertices, the 2D projection of the human body mesh vertices in the image is obtained. 2D =projection(verts) 3D ,cam).

[0106] Step 204: Determine the vertex visibility state based on the projection coordinates and the segmentation mask.

[0107] For each vertex, if its projected coordinates are located in the region with a segmentation mask of 1 and it is not occluded by other objects, it is marked as visible; otherwise, it is marked as invisible.

[0108] Specifically, based on the 2D coordinates of the human body mesh vertices and the mask output of the detection model, the color information of the visible parts of the human body in the image frame is mapped onto the human body mesh. Assume the 2D coordinates of the j-th vertex projected onto the image are verts. 2D,j =[h j ,w j ]: If the mask value at this coordinate is mask[h j ,w j If the value is 1, then the color value V at that coordinate in the video frame will be... i [h j ,w jFor the j-th vertex of the human body mesh, if the mask value mask[h j ,w j at this coordinate is 0, then set the j-th vertex of the human body mesh to be invisible.

[0109] In some embodiments, the vertex color value can be determined according to the color values of the vertex in the visible state in other frames within the set time window based on the current frame in the following manner.

[0110] In the case of real-time processing, the vertex color value is determined according to the color value of the vertex in the visible state that appeared most recently before the current frame, that is, the forward nearest visible frame is used to determine the vertex color value.

[0111] Specifically, for the vertex v that is invisible in the current frame t, find the frame k (k < t) closest to t before the t-th frame, in which the vertex v is visible, and use the color value of v in this frame k.

[0112] In the case of offline processing, the vertex color value can be determined according to the color value of the vertex in the visible state that appeared most recently after the current frame, that is, the vertex color value is determined according to the backward nearest visible frame.

[0113] Specifically, for the vertex v that is invisible in the current frame t, find the frame m (m > t) closest to t after the t-th frame, in which the vertex v is visible, and use the color value of v in this frame m.

[0114] The vertex color value can also be determined according to the average value of the color values of the vertex in the visible state that appear in all frames within the set time window.

[0115] Specifically, for the vertex v that is invisible in the current frame t, within a fixed time window [t - N, t + M] centered on t (or only including before t), collect the color values in all frames in which the vertex v is in the visible state, and calculate the average value of the RGB channels of these color values as the color value of v in the t-th frame.

[0116] In the embodiments of the present disclosure, through multi-frame information fusion, the single-frame visual limitation can be broken through, a time-dimensional observation model can be established, and the robustness, accuracy, and stability of the invisible vertex color value estimation can be improved.

[0117] In some embodiments, the mapping of the coordinates and color values of each mesh vertex to a UV map with a fixed resolution includes: mapping the three-dimensional mesh vertices to the UV plane through a predefined mapping function, where the UV plane is discretized into a rasterized map; for each raster position, obtaining at least one vertex mapped to this position; and filling the raster according to the color value corresponding to the at least one vertex.

[0118] In some cases, such as the folds of a judo uniform or the curved surface of a boxing glove, there may be multiple vertices that map to the same grid. In such cases, a certain strategy is needed to determine the color value to be filled.

[0119] In one example, a distance-first strategy can be used, which involves filling the grid with the color value of the vertex whose 2D projection position is closest to the grid center. This strategy is suitable for scenarios with high computational efficiency requirements.

[0120] In another example, a distance-weighted strategy can be used, where the blended color value is used to fill the raster based on the distance weight between the 2D projection position of each vertex and the center of the raster. This strategy is suitable for scenarios with high requirements for texture quality and visual continuity.

[0121] Figure 4 This is a block diagram of a contestant identification device provided in an exemplary embodiment. Figure 4 As shown, the device includes:

[0122] The first acquisition unit 401 is used to acquire a video image sequence containing a motion scene of opposing sides, wherein the clothing colors of the opposing sides conform to the regulations of the motion scene;

[0123] The second acquisition unit 402 is used to acquire the three-dimensional human body mesh of each contestant and the interfering person from the image frames in the sequence, and the three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices.

[0124] Extraction unit 403 is used to extract vertex color values ​​from the video image based on the vertex visibility state and mesh projection relationship;

[0125] The mapping unit 404 is used to map the color values ​​of each mesh vertex to a UV map of fixed resolution through a predefined vertex index, thereby generating a pose-invariant human texture representation.

[0126] The classification unit 405 is used to input the UV map into a pre-trained classification model to obtain a classification result, wherein the classification model is trained based on the UV map dataset in the motion scene.

[0127] The identification unit 406 is used to identify the two opposing players in the image frame based on the classification result.

[0128] In some embodiments, the second acquisition unit is specifically used for:

[0129] For the image frame, the human detection bounding boxes and segmentation masks of each contestant and interfering person are obtained according to the human detection algorithm or tracking algorithm.

[0130] The human body region image is cropped according to the human body detection box, and the human body region image is processed by a pre-trained human pose estimation model to obtain the vertex 3D coordinates and camera parameters.

[0131] The three-dimensional coordinates of the vertices are projected onto a two-dimensional image plane using the camera parameters.

[0132] Vertex visibility status is determined based on projection coordinates and segmentation mask.

[0133] In some embodiments, when the second acquisition unit is used to determine the vertex visibility state based on the projection coordinates and the segmentation mask, it is specifically used for:

[0134] For each vertex, if the projected coordinates are located in the region with a segmentation mask of 1 and are not occluded by other targets, it is marked as visible.

[0135] Otherwise, mark it as invisible.

[0136] In some embodiments, the extraction unit is specifically used for:

[0137] For vertices whose visibility is visible, extract the color value of the corresponding position of the two-dimensional projection coordinates from the video image;

[0138] For vertices whose visibility is not visible, set the color value to zero, or determine the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame's time window.

[0139] In some embodiments, when the extraction unit determines the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame time window, it is specifically used for:

[0140] In real-time processing, the vertex color value is determined based on the color value of the vertex in the visible state most recently before the current frame.

[0141] In offline processing, the vertex color value is determined based on the color value of the vertex in its most recent visible state after the current frame; or

[0142] The vertex color value is determined based on the average color value of all visible vertices that appear within a set time window.

[0143] In some embodiments, the mapping unit is specifically used for:

[0144] The vertices of the 3D mesh are mapped to the UV plane using a predefined mapping function, wherein the UV plane is discretized into a rasterized texture.

[0145] For each grid position, obtain at least one vertex mapped to that position;

[0146] The grid is filled according to the color value corresponding to the at least one vertex.

[0147] In some embodiments, when the mapping unit is used to fill the raster according to the color value corresponding to the at least one vertex, it is specifically used for:

[0148] Based on the distance weights between the two-dimensional projection position of each vertex and the center of the grid, a blended color value is generated to fill the grid.

[0149] The grid is filled with the color value of the vertex whose two-dimensional projection position is closest to the grid center.

[0150] In some embodiments, the apparatus further includes a training unit for:

[0151] Construct a UV map dataset that includes three categories: opposing sides and interfering personnel;

[0152] The classification model is trained using the UV mapping dataset.

[0153] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment. Please refer to... Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, memory 508, and non-volatile memory 510, and may also include other hardware required for business operations. One or more embodiments of this specification can be implemented in software, such as the processor 502 reading the corresponding computer program from the non-volatile memory 510 into memory 508 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0154] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0155] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0156] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0157] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0160] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0161] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0162] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A contestant identification method, characterized in that, include: Acquire a video image sequence containing a motion scene of opposing sides, wherein the clothing colors of the opposing sides conform to the regulations of the motion scene; The three-dimensional human body mesh of each contestant and the interfering person is obtained from the image frames in the sequence. The three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices. Based on the vertex visibility state and mesh projection relationship, vertex color values ​​are extracted from the video image; The color values ​​of each mesh vertex are mapped to a UV map of fixed resolution using a predefined vertex index to generate a pose-invariant human texture representation. The UV map is input into a pre-trained classification model to obtain a classification result, wherein the classification model is trained based on the UV map dataset in the motion scene; The opposing players in the image frame are identified based on the classification results.

2. The method according to claim 1, characterized in that, The step of obtaining the three-dimensional human body mesh of each contestant and interfering person from the image frames in the sequence includes: For the image frame, the human detection bounding boxes and segmentation masks of each contestant and interfering person are obtained according to the human detection algorithm or tracking algorithm. The human body region image is cropped according to the human body detection box, and the human body region image is processed by a pre-trained human pose estimation model to obtain the vertex 3D coordinates and camera parameters. The three-dimensional coordinates of the vertices are projected onto a two-dimensional image plane using the camera parameters. Vertex visibility status is determined based on projection coordinates and segmentation mask.

3. The method according to claim 2, characterized in that, The step of determining vertex visibility state based on projection coordinates and segmentation mask includes: For each vertex, if the projected coordinates are located in the region with a segmentation mask of 1 and are not occluded by other targets, it is marked as visible. Otherwise, mark it as invisible.

4. The method according to claim 1, characterized in that, Extracting vertex color values ​​from the video image based on the vertex visibility state and mesh projection relationship includes: For vertices whose visibility is visible, extract the color value of the corresponding position in the two-dimensional projection coordinates from the video image; For vertices whose visibility is not visible, set the color value to zero, or determine the vertex color value based on the color value of the vertex in the visible state in other frames within the current frame's time window.

5. The method according to claim 4, characterized in that, The vertex color value is determined based on the color values ​​of the vertex in the visible state in other frames within the current frame's time window, including: In real-time processing, the vertex color value is determined based on the color value of the vertex in the visible state most recently before the current frame. In offline processing, the vertex color value is determined based on the color value of the vertex in its most recent visible state after the current frame; or The vertex color value is determined based on the average color value of all visible vertices that appear within a set time window.

6. The method according to claim 4, characterized in that, The process of mapping the coordinates and color values ​​of each mesh vertex to a UV map of fixed resolution includes: The vertices of the 3D mesh are mapped to the UV plane using a predefined mapping function, wherein the UV plane is discretized into a rasterized texture. For each grid position, obtain at least one vertex mapped to that position; The grid is filled according to the color value corresponding to the at least one vertex.

7. The method according to claim 6, characterized in that, The step of filling the raster according to the color value corresponding to the at least one vertex includes at least one of the following: Based on the distance weights between the two-dimensional projection position of each vertex and the center of the grid, a blended color value is generated to fill the grid. The grid is filled with the color value of the vertex whose two-dimensional projection position is closest to the grid center.

8. The method according to claim 1, characterized in that, The method further includes: Construct a UV map dataset that includes three categories: opposing sides and interfering personnel; The classification model is trained using the UV mapping dataset.

9. A contestant identification device, characterized in that, include: The first acquisition unit is used to acquire a video image sequence containing a motion scene of opposing sides, wherein the clothing colors of the opposing sides conform to the regulations of the motion scene; The second acquisition unit is used to acquire the three-dimensional human body mesh of each contestant and the interfering person from the image frames in the sequence, and the three-dimensional human body mesh includes the three-dimensional coordinates of the vertices and the visibility status of the vertices. An extraction unit is used to extract vertex color values ​​from the video image based on the vertex visibility state and the mesh projection relationship; The mapping unit is used to map the color values ​​of each mesh vertex to a UV map of fixed resolution through a predefined vertex index, thereby generating a pose-invariant human texture representation. A classification unit is used to input the UV map into a pre-trained classification model to obtain a classification result, wherein the classification model is trained based on the UV map dataset in the motion scene; The identification unit is used to identify the two opposing players in the image frame based on the classification result.

10. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-8 by executing the executable instructions.

Images