Image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic device and storage medium, wherein the processing method comprises: obtaining at least two images to be processed with target objects corresponding to the current moment, and each model to be tracked corresponding to the previous moment of the current moment; for any model to be tracked, among the target objects in the image to be processed, determining the target object that successfully matches any model to be tracked; generating a current model based on the target objects that have not successfully matched the model to be tracked corresponding to each previous moment; determining the matching result between the current model and the historical model; and generating an image processing result based on the matching result. The disclosed embodiment enables the current model to still achieve tracking and matching in the event of image frame loss, which is conducive to improving the practical application value of human posture reconstruction.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the development of image processing, human posture reconstruction technology has gradually attracted people's attention. Human posture reconstruction technology can build a three-dimensional model of the target object based on the image with the target object, and can be applied to scenes such as movie special effects, target positioning, and human-computer interaction. Therefore, how to better reconstruct human posture is a technical problem that developers urgently need to solve. Summary of the invention

[0003] The present disclosure proposes a technical solution for image processing.

[0004] According to one aspect of the present disclosure, a method for processing an image is provided, the method comprising: obtaining at least two images to be processed with target objects corresponding to a current moment and each model to be tracked corresponding to a moment before the current moment; for any model to be tracked, determining, among the target objects in the images to be processed, a target object that successfully matches any model to be tracked; generating a current model based on target objects that have not successfully matched any model to be tracked corresponding to each previous moment; determining a matching result between the current model and a historical model; wherein the historical model is a model to be tracked within a preset time period before the current moment; and generating an image processing result based on the matching result.

[0005] In a possible implementation, for any model to be tracked, determining a target object that successfully matches any model to be tracked among the target objects in the image to be processed, includes: for any model to be tracked, combining target objects in different images to be processed to obtain a first object group corresponding to any model to be tracked; screening the first object group and generating a three-dimensional model corresponding to each screened first object group; determining a first model that successfully matches any model to be tracked among the three-dimensional models corresponding to the screened first object group; and using the target object corresponding to the first model as the target object that successfully matches any model to be tracked.

[0006] In a possible implementation, generating the current model based on the target object that has not been successfully matched with the model to be tracked corresponding to each previous moment includes: determining a reference image and a non-reference image in the image to be processed; taking the target object in the reference image that has not been successfully matched with the model to be tracked corresponding to each previous moment as the object to be matched, and taking the target object in the non-reference image that has not been successfully matched with the model to be tracked corresponding to each previous moment as the first object; for any object to be matched, combining the first objects in different non-reference images to obtain a second object group corresponding to any object to be matched; screening the second object group, and generating a three-dimensional model corresponding to each screened second object group based on any object to be matched and the second object group; and determining, in the three-dimensional models corresponding to the screened second object group, the current model that has successfully matched with any object to be matched.

[0007] In a possible implementation, the target objects in different images to be processed are combined to obtain a first object group corresponding to any model to be tracked, including: for any image to be processed, a target object in any image to be processed that has not been successfully matched with the model to be tracked is determined as a first unmatched object, and based on the distance between the first unmatched object and any model to be tracked, it is determined whether the first unmatched object is a second object; and the second objects in different images to be processed are combined to obtain a first object group corresponding to any model to be tracked.

[0008] In a possible implementation, the second object is a physical object or a non-physical object; wherein the non-physical object is used to represent an object that is not captured in the image to be processed, and the physical object is used to represent a target object actually captured in the image to be processed; and the screening of the first object group includes: screening a first object group having at least two physical objects as the screened first object group.

[0009] In a possible implementation, combining the first objects in different non-reference images to obtain the second object group corresponding to any object to be matched includes: for any non-reference image, determining the first object in any non-reference image that has not been successfully matched with the object to be matched as a second unmatched object, and determining whether the second unmatched object is a third object based on the distance between the second unmatched object and any object to be matched; and combining the third objects in different non-reference images to obtain the second object group corresponding to any object to be matched.

[0010] In a possible implementation, the third object is a physical object or a non-physical object; and screening the second object group includes: screening a second object group having at least two physical objects as the screened second object group.

[0011] In one possible implementation, generating an image processing result based on the matching result includes: using the model in the current model that successfully matches the historical model and the first model as the model to be tracked corresponding to the current moment; wherein the model to be tracked corresponding to the current moment is used to match the target object corresponding to the next moment after the current moment.

[0012] In a possible implementation, generating an image processing result based on the matching result includes: when it is determined that both the current model and the historical model have failed to match and a preset condition is met, determining that the model to be tracked corresponding to the current moment includes the current model, and generating an image processing result; wherein the preset condition includes at least one of the following: the current model meets a preset ratio, the number of models to be tracked corresponding to the current moment is less than or equal to a preset number, and the distance between the current model and any model to be tracked corresponding to the current moment is greater than a preset distance.

[0013] In a possible implementation, generating an image processing result based on the matching result also includes: for any model to be tracked corresponding to the current moment, optimizing the model to be tracked corresponding to any current moment based on the model to be tracked corresponding to the previous moment that successfully matches the model to be tracked corresponding to the current moment or the historical model; generating an image processing result based on the matching conditions between the models to be tracked corresponding to different moments and each model to be tracked corresponding to the current moment after optimization.

[0014] In a possible implementation, the obtaining of at least two images to be processed corresponding to the current moment and having the target object includes: obtaining at least two initial images corresponding to the current moment; for any initial image, determining all initial objects in the initial image; determining whether any initial object is the target object based on the position of any initial object in the initial image; and using the initial image with the target object as the image to be processed.

[0015] In a possible implementation, the initial image includes a partial area of ​​a sports field; the initial object includes at least one of an on-field athlete object, a substitute athlete object, and a spectator object; the target object is an on-field athlete object; and determining whether any initial object is a target object based on the position of any initial object in any initial image includes: determining whether any initial object is an on-field athlete object based on the position of a foot key point of any initial object in any initial image projected onto the sports field.

[0016] According to one aspect of the present disclosure, there is provided an image processing device, the processing device comprising: a data acquisition module, used to acquire at least two images to be processed with target objects corresponding to a current moment, and each model to be tracked corresponding to a moment before the current moment; a target object determination module, used to determine, for any model to be tracked, a target object that successfully matches any model to be tracked among the target objects in the images to be processed; a current model generation module, used to generate a current model based on target objects that have not successfully matched any model to be tracked corresponding to each previous moment; a matching result determination module, used to determine the matching result between the current model and a historical model; wherein the historical model is a model to be tracked within a preset time period before the current moment; and an image processing result generation module, used to generate an image processing result based on the matching result.

[0017] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned processing method.

[0018] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned processing method is implemented.

[0019] In the disclosed embodiment, at least two images to be processed with target objects corresponding to the current moment and each model to be tracked corresponding to the previous moment of the current moment can be obtained, and then for any model to be tracked, among the target objects in the image to be processed, the target object that successfully matches any model to be tracked is determined, and then the current model is generated based on the target objects that have not successfully matched the model to be tracked corresponding to each previous moment, and the matching result between the current model and the historical model is determined, and finally the image processing result is generated based on the matching result. The disclosed embodiment can reconstruct the current model corresponding to the target object based on multiple images to be processed, and track and match the current model with the historical model before the current moment and within a preset time length in the time series, so that the current model can still achieve tracking and matching in the event of image frame loss, which is conducive to improving the practical application value of human posture reconstruction.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0022] Figure 1 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown.

[0023] Figure 2 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown.

[0024] Figure 3 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown.

[0025] Figure 4 A reference schematic diagram is shown of an image processing method provided by an embodiment of the present disclosure.

[0026] Figure 5 A block diagram of an image processing device provided according to an embodiment of the present disclosure is shown.

[0027] Figure 6 A block diagram of an electronic device provided according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0029] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0030] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0031] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0032] In the related art, human posture reconstruction is usually performed in the following ways: 1. Directly map the target object under multiple perspectives into the three-dimensional space, and fit the coordinates of the skeleton key points of the target object through the machine learning model. This method relies on a large amount of annotated three-dimensional data, and the labor cost is high. In addition, the machine learning model needs to process the characteristics of the target object in the three-dimensional space, so this method has the problem of slow processing speed of the human posture reconstruction process, especially when there are multiple target objects. This problem is particularly serious. 2. Focus on the human posture reconstruction of a single frame image, and then track the target objects that appear in the entire shooting time series. However, this method has too much calculation when there are multiple target objects, and also has the problem of slow processing speed of the human posture reconstruction process. In addition, when the target object appears in the time series, the tracking will be disconnected when the frame is lost, which is not conducive to adapting to the actual application scenario of human posture reconstruction.

[0033] In view of this, the embodiment of the present disclosure provides an image processing method, which can obtain at least two images to be processed with target objects corresponding to the current moment, and each model to be tracked corresponding to the previous moment of the current moment, and then for any model to be tracked, among the target objects in the image to be processed, determine the target object that successfully matches any model to be tracked, and then generate a current model based on the target objects that have not successfully matched the model to be tracked corresponding to each previous moment, determine the matching result between the current model and the historical model, and finally generate an image processing result based on the matching result. The embodiment of the present disclosure can reconstruct the current model corresponding to the target object based on multiple images to be processed, and track and match the current model with the historical model before the current moment and within a preset time length in the time series, so that the current model can still achieve tracking and matching in the event of image frame loss, which is conducive to improving the practical application value of human posture reconstruction.

[0034] In a possible implementation, the processing method provided by the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. For any electronic device, the processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0035] See also Figure 1 As shown, Figure 1 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown. Figure 1 As shown, the processing method may include:

[0036] Step S100, obtaining at least two images to be processed with the target object corresponding to the current moment, and each model to be tracked corresponding to the moment before the current moment. Exemplarily, the target object may be a human object, and the image to be processed may be collected by multiple cameras at different viewing angles connected to the electronic device by wire or wirelessly. The setting position of the camera is not limited in the embodiment of the present disclosure, and can be set by the user, as long as the detection area desired by the user can be captured. In one example, after the camera is installed, the intrinsic parameters and extrinsic parameters of each camera may be adjusted so that different cameras can share the same world coordinate system. In other words, the coordinate positions of the same target object captured by different cameras in the world coordinate system are the same or similar, so as to improve the accuracy of the subsequent generation of the current model, thereby improving the accuracy of the image processing results. The method for adjusting the camera extrinsic parameters may refer to the relevant technology, and the embodiment of the present disclosure will not be repeated here. Exemplarily, the above-mentioned image to be processed can be collected in real time by the above-mentioned camera. In other words, the processing method provided by the embodiment of the present disclosure can be applied to real-time video streams, that is, each camera can collect a real-time video stream, and the image to be processed collected by each camera can be determined by the corresponding time information (for example: the above-mentioned current time, the previous time) to determine the position of the image to be processed in the real-time video stream collected by the camera. Exemplarily, the model to be tracked corresponding to the previous time can be a three-dimensional model obtained by fusing the images of the area where the same target object is located in at least two images to be processed corresponding to the previous time (which can be collected by at least two cameras), which will be described in detail later.

[0037] In a possible implementation, the acquisition of at least two images to be processed corresponding to the current moment with the target object may include: acquiring at least two initial images corresponding to the current moment, the initial images may be original images including human objects collected by different cameras. Then, for any initial image, all initial objects in the initial image are determined. Exemplarily, the initial objects may be all human objects in the initial image, which may be obtained by a human detection algorithm in the relevant technology, and then, according to the position of any initial object in the initial image, whether the initial object is the target object is determined. Exemplarily, the electronic device may establish a mapping relationship between the actual area plane (e.g., the ground) and the shooting plane of the camera through a homography matrix (i.e., Homography matrix) in the relevant technology, so that the position of the initial object in the initial image can be mapped to the actual area plane, and then the position of the initial object in the actual area plane is obtained, which may be expressed as a coordinate information. Then, the electronic device may compare the coordinate information with a preset detection area plane (e.g., various detection areas defined by humans, such as the sports area surrounded by the field lines of various courts) to determine whether the initial object is located in the detection area plane. If the initial object is located in the plane of the detection area, it can be determined that the initial object is the target object, and the initial image with the target object is used as the image to be processed.

[0038] In a possible implementation, the initial image includes a partial area of ​​the sports field; the initial object includes at least one of an on-field athlete object, a substitute athlete object, and an audience object; the target object is an on-field athlete object; the determining whether any of the initial objects is a target object according to the position of any of the initial objects in the initial image includes: determining whether any of the initial objects is an on-field athlete object according to the position of the foot key points of any of the initial objects in the initial image projected on the sports field. In combination with the actual scene, the position of the initial object can be expressed as the coordinates of the foot key points of the initial object in the actual area plane. Under normal circumstances, the foot key points of the athlete will not be too far away from the actual area plane, so the representativeness of the initial object position can be improved by calculating the foot key points, which is conducive to improving the accuracy of subsequent image processing results. The disclosed embodiment can filter the initial object so that the filtered target object is more in line with the user's human posture reconstruction requirements. In combination with the actual scene, the smart basketball court is taken as an example here. In this example, the detection area plane is the sports area (i.e., the active area of ​​the athlete in the basketball court during the game, i.e., the area surrounded by the basketball court line), and the actual area plane may include the sports area, the substitute player area, and part of the audience area. Then the initial objects may include athletes, substitutes, and spectators in the game, and the electronic device can use the athletes in the game as the target object to reconstruct the human posture. In combination with the actual scenario, users usually pay attention to the athletes in the game, so the human posture reconstruction can only be targeted at the athletes in the game to reduce the computing pressure of the electronic device. In some examples, the audience area or the substitute area can also be used as part of the detection area plane, that is, the human posture reconstruction can be performed not only for the athletes in the game, but also for the audience or substitutes to create a game atmosphere. The developer can flexibly set it according to the computing power level of the electronic device, and the disclosed embodiment is not limited here.

[0039] Continue reading Figure 1 , step S200, for any model to be tracked, among the target objects in the image to be processed, determine the target object that successfully matches the model to be tracked. Exemplarily, the above matching success can be determined by calculating the similarity between the three-dimensional model generated based on the same target object in different images to be processed and the model to be tracked, which is not limited in the embodiments of the present disclosure.

[0040] See also Figure 2 As shown, Figure 2 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown. Figure 2 As shown, in a possible implementation, step S200 may include:

[0041] Step S210, for any model to be tracked, the target objects in different images to be processed are combined to obtain the first object group corresponding to any model to be tracked. Exemplarily, the same target object may correspond to different two-dimensional tracking numbers, detection frame coordinates, skeleton key point coordinates, etc. in different images to be processed (which can be obtained by the object detection algorithm in the relevant technology, and the embodiments of the present disclosure are not repeated here), and then the target objects with different two-dimensional tracking numbers are combined to obtain the above-mentioned first object group. Exemplarily, the above-mentioned first object group is used to realize the fusion of the target object at the current moment, and the tracking and matching between the three-dimensional model fused at the current moment and the model to be tracked at the previous moment. In combination with the actual application scenario, taking the smart basketball court as an example, after the electronic device obtains the image to be processed corresponding to the current moment, the electronic device can generate the athlete model at the current moment, and track and match it with the athlete model at the previous moment. If the tracking and matching is successful, the electronic device can consider the two models to be the human body posture reconstruction results of the same athlete at different moments. In an example, the electronic device can also mark the above-mentioned two models with the same logo to facilitate audience identification, and the embodiments of the present disclosure are not limited here.

[0042] In a possible implementation, step S210 may include: for any image to be processed, determining the target object in any image to be processed that is not successfully matched with the model to be tracked as the first unmatched object, and determining whether the first unmatched object is the second object according to the distance between the first unmatched object and the model to be tracked. Exemplarily, the coordinates of the target object and the model to be tracked in the detection area can be determined according to the above homography matrix, and then the offset of the two coordinates is calculated. When the offset between the target object and the model to be tracked is less than the preset offset, the target object can be determined to be the second object. Finally, the second objects in different images to be processed are combined to obtain the first object group corresponding to any model to be tracked. Combined with the actual scene, when the calibration error of the camera internal parameters and external parameters is small, after mapping the image position of the same athlete in different images to be processed to the above detection area plane, the detection area coordinates corresponding to different images to be processed should be similar, so the embodiment of the present disclosure can allow the combination between the relatively close target objects under different viewing angles to increase the probability that each first object group is the same athlete. The embodiment of the present disclosure can improve the correlation between the second object and the model to be tracked by screening out the second object, which is beneficial to improving the accuracy of the current model generated based on the second object. In addition, since the second object is screened out, the embodiment of the present disclosure can reduce the computing power loss of the electronic device, which is beneficial to improving the real-time response capability of the electronic device when processing real-time video streams.

[0043] In a possible implementation, the second object may be a physical object or a non-physical object. Wherein, the non-physical object is used to represent an object not captured in the image to be processed, and the physical object is used to represent a target object actually captured in the image to be processed. The disclosed embodiment enables the electronic device to adapt to the actual shooting scene by combining physical objects or non-physical objects in different images. For example: in the actual shooting scene, it may happen that the camera cannot capture each target object, such as: basketball game recording scene, because the detection area in the scene is large, there may be a situation where a single camera does not capture the athlete comprehensively (for example: in order to improve the shooting clarity, the user may set each camera to capture part of the detection area). For example: camera 1 actually captures athletes A, B, C (that is, the physical objects in the image to be processed captured by camera 1), camera 2 actually captures athletes D, E, and camera 3 actually captures athletes F, G. Each camera can correspond to a non-physical object K, and the distance between the non-physical object K and the model to be detected can be set to a predetermined value, such as 0, so that the non-physical object will not be screened out in the distance comparison above. When combining the second objects in different images to be processed, the embodiments of the present disclosure allow the following combinations: A, K, F, K, D, F, A, D, K, etc. That is, the first object group generated by the embodiments of the present disclosure takes into account the situation where the camera does not capture the physical object, thereby improving the generation accuracy of the three-dimensional model corresponding to the subsequent first object group, which is beneficial to improving the tracking and matching accuracy of the three-dimensional model and the model to be tracked. Continuing with the above example, if A and F are actually the same athlete, and A and D and E are not the same athlete, the accuracy of the three-dimensional model generated based on A, K, F is higher than that of the three-dimensional model generated based on A, D, F or A, E, F. In other words, the embodiments of the present disclosure do not need to forcibly combine actually different athletes.

[0044] Continue reading Figure 2 , step S220, filtering the first object group, and generating a three-dimensional model corresponding to each filtered first object group. Exemplarily, based on at least two second objects corresponding to each first object group, a three-dimensional reconstruction algorithm (such as a triangulation algorithm) in the related technology can be used to generate a three-dimensional model corresponding to each first object group, which is not described in detail in the embodiments of the present disclosure. In one example, if the second object is a physical object or a non-physical object, the electronic device can filter the first object group with at least two physical objects as the filtered first object group so that the three-dimensional reconstruction algorithm can be correctly executed. In addition, the computing power of the electronic device can be saved.

[0045] Step S230, in the three-dimensional models corresponding to the first object group after the screening, determine the first model that successfully matches any of the models to be tracked. Exemplarily, based on each three-dimensional model, any of the models to be tracked, and the regional image of the target object corresponding to each three-dimensional model, the error sum in various related technologies corresponding to each three-dimensional model can be determined, and the error sum is negatively correlated with the similarity, such as: reprojection error, three-dimensional Euclidean distance error, two-dimensional tracking number error, Welch error, etc., which are not limited in the embodiments of the present disclosure. Then, when the minimum error sum is less than a preset error threshold, the corresponding three-dimensional model can be used as the first model. Exemplarily, the first model is obtained based on the model to be tracked at the previous moment, so the first model can be regarded as the tracking and matching result of the model to be tracked that successfully matches the first model. For example, the embodiment of the present disclosure can make the first model and the model to be tracked share a three-dimensional model number (that is, the electronic device believes that the two models belong to the same target object at different moments in the actual scene) to show the association between the two models.

[0046] Step S240: taking the target object corresponding to the first model as the target object that successfully matches any of the models to be tracked. In combination with actual scenes, the target object that successfully matches any of the models to be tracked is the on-field athlete captured in different images to be processed.

[0047] Continue reading Figure 1 , step S300, generating a current model according to a target object that has not been successfully matched with any of the to-be-tracked models corresponding to each previous moment. The current model can be obtained based on the target object according to a 3D reconstruction algorithm in the related art, which will not be described in detail in the embodiment of the present disclosure.

[0048] See also Figure 3 As shown, Figure 3 A flowchart of an image processing method provided according to an embodiment of the present disclosure is shown. Figure 3 As shown, in a possible implementation, step S300 may include:

[0049] Step S310, determining the reference image and the non-reference image in the image to be processed. For example, the developer can select any reference camera from the cameras, and then use the image to be processed taken by the reference camera as the reference image, and use the image to be processed taken by the non-reference camera as the non-reference image.

[0050] Step S320, taking the target object in the reference image that has not been successfully matched with the to-be-tracked model corresponding to each previous moment as the to-be-tracked object, and taking the target object in the non-reference image that has not been successfully matched with the to-be-tracked model corresponding to each previous moment as the first object.

[0051] Step S330 : for any object to be matched, first objects in different non-reference images are combined to obtain a second object group corresponding to the object to be matched.

[0052] In a possible implementation, step S330 may include: for any non-reference image, determining the first object in any non-reference image that is not successfully matched with the object to be matched as the second unmatched object, and determining whether the second unmatched object is a third object according to the distance between the second unmatched object and any object to be matched. Exemplarily, the coordinates of the first object and the object to be matched in the detection area can be determined according to the above homography matrix, and then the offset of the coordinates of the two can be calculated. When the offset between the first object and the object to be matched is less than the preset offset, the first object can be determined to be the third object. Finally, the third objects in different non-reference images are combined to obtain the second object group corresponding to any object to be matched. The embodiment of the present disclosure can improve the correlation between the third object and the object to be matched by screening out the third object, which is conducive to improving the accuracy of the three-dimensional model generated based on the third object. In addition, since the third object is screened out, the embodiment of the present disclosure can reduce the computing power loss of the electronic device, which is conducive to improving the real-time response capability of the electronic device when processing real-time video streams.

[0053] In a possible implementation, the third object may be a physical object or a non-physical object. The disclosed embodiment can make the electronic device adaptable to the actual shooting scene by combining physical objects or non-physical objects in different non-reference images. That is, the second object group generated by the disclosed embodiment takes into account the situation that the camera does not capture the physical object, thereby improving the generation accuracy of the three-dimensional model corresponding to the subsequent second object group, which is conducive to improving the generation accuracy of the current model obtained subsequently.

[0054] Step S340, filtering the second object group, and generating a three-dimensional model corresponding to each filtered second object group based on any of the objects to be matched and the second object group. Exemplarily, the three-dimensional model corresponding to each second object group can be generated based on at least two third objects corresponding to each second object group through a three-dimensional reconstruction algorithm (such as a triangulation algorithm) in the relevant technology, which is not described in detail in the embodiments of the present disclosure. In one example, if the third object is a physical object or a non-physical object, the electronic device can filter the second object group with at least two physical objects as the filtered second object group, so that the three-dimensional reconstruction algorithm can be correctly executed, while saving the computing power of the electronic device.

[0055] Step S350: Determine, in the three-dimensional models corresponding to the filtered second object group, a current model that successfully matches any of the objects to be matched.

[0056] Exemplarily, based on each three-dimensional model, the regional image of any of the above-mentioned objects to be matched, and the regional image of the target object corresponding to each three-dimensional model, the error sum in various related technologies corresponding to each three-dimensional model, such as: reprojection error, Welch error, etc., is determined, and the embodiments of the present disclosure are not limited here. Then, when the minimum error sum is less than the preset error threshold, the corresponding three-dimensional model can be used as the above-mentioned current model. Exemplarily, the above-mentioned current model is obtained based on the object to be matched at the current moment, so the current model can be regarded as the multi-perspective fusion result of the object to be matched. Combined with actual application scenarios, there are usually situations where athletes are replaced in sports competitions, that is, since the current model has not been successfully matched with the model to be tracked corresponding to the previous moment, the electronic device can regard the current model as a new athlete or an athlete in the case of frame loss.

[0057] Continue reading Figure 1 , step S400, determining the matching result between the current model and the historical model. Wherein, the historical model is the model to be tracked before the current moment and within a preset time length. Exemplarily, the above matching result can be compared by comparing the similarity between the historical model and the current model. Exemplarily, the geometric features and appearance features of the two can be calculated by a related algorithm or model, and compared. When the similarity between the two is higher than the preset similarity, it is determined that the current model and the historical model are successfully matched. The disclosed embodiment can handle the tracking of target objects in the case of continuous frame loss to improve the tracking accuracy of the target object in the real-time video stream.

[0058] Step S500, according to the matching result, generates image processing result. In conjunction with actual application scenario, if matching fails, the electronic device thinks that the current model is a new athlete, and a new logo can be used to mark it. If matching is successful, the electronic device thinks that the current model is an athlete who has played under the frame loss situation, and the original logo of the athlete who has played can be used (such as the logo corresponding to the historical model of successful matching) to mark it. The above-mentioned image processing result can be expressed as a mark, such as 10 athletes at a certain moment, then each athlete's annotation frame can be generated, and the athlete's personal information (such as name) is generated near the annotation frame, and at the next moment, the same athlete's annotation frame position can be changed, but the personal information can remain the same, and the human posture reconstruction and tracking of multiple athletes can be completed.

[0059] In a possible implementation, step S500 includes: taking the model in the current model that successfully matches the historical model and the first model as the model to be tracked corresponding to the current moment; wherein the model to be tracked corresponding to the current moment is used to match the target object corresponding to the next moment of the current moment. In combination with the actual scene, if the camera is shooting a real-time video stream, the model to be tracked corresponding to the current moment can be regarded as an athlete in the game, and when the camera captures a new image, the current moment of the camera will be updated, that is, the model to be tracked corresponding to the current moment will be regarded as the model to be tracked corresponding to the previous moment, and the image to be processed corresponding to the next moment will be regarded as the image to be processed corresponding to the current moment, and the real-time image processing process will continue.

[0060] In a possible implementation, step S500 may include the following steps: when it is determined that the current model and the historical model both fail to match and meet the preset conditions, determine that the model to be tracked corresponding to the current moment includes the current model, and generate an image processing result. Wherein, the preset conditions include at least one of the following: the current model meets the preset ratio, the number of models to be tracked corresponding to the current moment is less than or equal to the preset number, and the distance between the current model and any model to be tracked corresponding to the current moment is greater than the preset distance. In combination with the actual application scenario, the preset ratio may be the human skeleton ratio, the preset number may be the total number of athletes who can play under the regulations in various competitions, and the preset distance is used to determine whether the current model is a ghost image that occurs during shooting, such as the calibration error of the internal and external parameters of the camera is too large, which leads to the reconstruction of multiple current models of the same athlete at the current moment. When the current model and the historical model both fail to match and meet the preset conditions, the electronic device may consider the current model as a new athlete and assign a new identifier to it.

[0061] In a possible implementation, step S500 may also include the following steps: for any model to be tracked corresponding to the current moment, according to the model to be tracked corresponding to the previous moment or the historical model that successfully matches the model to be tracked corresponding to the current moment, optimize the model to be tracked corresponding to the current moment. Exemplarily, the model to be tracked corresponding to the current moment can be optimized by the time series interpolation algorithm, the bone fixed length optimization algorithm, etc. in the relevant technology. Among them, the time series interpolation algorithm can be used to fill in the frame in the case of frame loss, so that the movement of the three-dimensional model at different moments is more coherent. The bone fixed length optimization can be used to maintain the body proportions between the three-dimensional models of the same target object at different moments, so that the body proportion change range of the same target object at different moments is more reasonable. Then, according to the matching situation between the models to be tracked corresponding to different moments and each model to be tracked corresponding to the current moment after optimization, the image processing result is generated.

[0062] Reference Figure 4 As shown, Figure 4 A reference schematic diagram of an image processing method provided by an embodiment of the present disclosure is shown, Figure 4 As shown, in combination with actual application scenarios, the electronic device can capture images to be processed from different perspectives through different cameras, and then extract the target object (i.e. Figure 4 The multi-view players in the image are then converted into the 3D model corresponding to the target object (i.e. Figure 4 The current frame player) and the model to be tracked corresponding to the previous moment (i.e. Figure 4 If the match is successful, the human body posture of the 3D model is reconstructed successfully and the tracking model corresponding to the previous moment is successfully tracked. If the match fails, the current model is generated based on the target object that has not been successfully matched, and the historical model (i.e. Figure 4 If the match is successful, the human body posture of the current model is successfully reconstructed and the historical model is successfully tracked. If the match fails, the current model is used as the model that failed to match. If the interval between the last time a historical model appeared and the current time is greater than a threshold, it is filtered out from the historical model (i.e. Figure 4 In this way, the electronic device can complete the cross-frame tracking and matching of the player in the current frame and the reconstruction of the human body posture.

[0063] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0064] In addition, the present disclosure also provides an image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any image processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0065] Figure 5 A block diagram of an image processing device provided according to an embodiment of the present disclosure is shown, Figure 5As shown, the processing device 100 includes: a data acquisition module 110, which is used to acquire at least two images to be processed with target objects corresponding to the current moment and each model to be tracked corresponding to the previous moment of the current moment. A target object determination module 120, which is used to determine, for any model to be tracked, the target object in the target object in the image to be processed that successfully matches any model to be tracked. A current model generation module 130, which is used to generate a current model based on the target object that has not been successfully matched with the model to be tracked corresponding to each previous moment. A matching result determination module 140, which is used to determine the matching result between the current model and the historical model. The historical model is the model to be tracked within a preset time period before the current moment. An image processing result generation module 150, which is used to generate an image processing result based on the matching result.

[0066] In a possible implementation, for any model to be tracked, determining a target object that successfully matches any model to be tracked among the target objects in the image to be processed, includes: for any model to be tracked, combining target objects in different images to be processed to obtain a first object group corresponding to any model to be tracked; screening the first object group and generating a three-dimensional model corresponding to each screened first object group; determining a first model that successfully matches any model to be tracked among the three-dimensional models corresponding to the screened first object group; and using the target object corresponding to the first model as the target object that successfully matches any model to be tracked.

[0067] In a possible implementation, generating the current model based on the target object that has not been successfully matched with the model to be tracked corresponding to each previous moment includes: determining a reference image and a non-reference image in the image to be processed; taking the target object in the reference image that has not been successfully matched with the model to be tracked corresponding to each previous moment as the object to be matched, and taking the target object in the non-reference image that has not been successfully matched with the model to be tracked corresponding to each previous moment as the first object; for any object to be matched, combining the first objects in different non-reference images to obtain a second object group corresponding to any object to be matched; screening the second object group, and generating a three-dimensional model corresponding to each screened second object group based on any object to be matched and the second object group; and determining, in the three-dimensional models corresponding to the screened second object group, the current model that has successfully matched with any object to be matched.

[0068] In a possible implementation, the target objects in different images to be processed are combined to obtain a first object group corresponding to any model to be tracked, including: for any image to be processed, a target object in any image to be processed that has not been successfully matched with the model to be tracked is determined as a first unmatched object, and based on the distance between the first unmatched object and any model to be tracked, it is determined whether the first unmatched object is a second object; and the second objects in different images to be processed are combined to obtain a first object group corresponding to any model to be tracked.

[0069] In a possible implementation, the second object is a physical object or a non-physical object; wherein the non-physical object is used to represent an object that is not captured in the image to be processed, and the physical object is used to represent a target object actually captured in the image to be processed; and the screening of the first object group includes: screening a first object group having at least two physical objects as the screened first object group.

[0070] In a possible implementation, combining the first objects in different non-reference images to obtain the second object group corresponding to any object to be matched includes: for any non-reference image, determining the first object in any non-reference image that has not been successfully matched with the object to be matched as a second unmatched object, and determining whether the second unmatched object is a third object based on the distance between the second unmatched object and any object to be matched; and combining the third objects in different non-reference images to obtain the second object group corresponding to any object to be matched.

[0071] In a possible implementation, the third object is a physical object or a non-physical object; and screening the second object group includes: screening a second object group having at least two physical objects as the screened second object group.

[0072] In one possible implementation, generating an image processing result based on the matching result includes: using the model in the current model that successfully matches the historical model and the first model as the model to be tracked corresponding to the current moment; wherein the model to be tracked corresponding to the current moment is used to match the target object corresponding to the next moment after the current moment.

[0073] In a possible implementation, generating an image processing result based on the matching result includes: when it is determined that both the current model and the historical model have failed to match and a preset condition is met, determining that the model to be tracked corresponding to the current moment includes the current model, and generating an image processing result; wherein the preset condition includes at least one of the following: the current model meets a preset ratio, the number of models to be tracked corresponding to the current moment is less than or equal to a preset number, and the distance between the current model and any model to be tracked corresponding to the current moment is greater than a preset distance.

[0074] In a possible implementation, generating an image processing result based on the matching result also includes: for any model to be tracked corresponding to the current moment, optimizing the model to be tracked corresponding to any current moment based on the model to be tracked corresponding to the previous moment that successfully matches the model to be tracked corresponding to the current moment or the historical model; generating an image processing result based on the matching conditions between the models to be tracked corresponding to different moments and each model to be tracked corresponding to the current moment after optimization.

[0075] In a possible implementation, the obtaining of at least two images to be processed corresponding to the current moment and having the target object includes: obtaining at least two initial images corresponding to the current moment; for any initial image, determining all initial objects in the initial image; determining whether any initial object is the target object based on the position of any initial object in the initial image; and using the initial image with the target object as the image to be processed.

[0076] In a possible implementation, the initial image includes a partial area of ​​a sports field; the initial object includes at least one of an on-field athlete object, a substitute athlete object, and a spectator object; the target object is an on-field athlete object; and determining whether any initial object is a target object based on the position of any initial object in any initial image includes: determining whether any initial object is an on-field athlete object based on the position of a foot key point of any initial object in any initial image projected onto the sports field.

[0077] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0078] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0079] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0080] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0081] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0082] The electronic device may be provided as a server or other form of device.

[0083] Figure 6 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 6 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0084] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0085] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0086] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0087] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0088] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0089] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0090] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0091] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0092] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0093] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0094] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0095] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0096] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0097] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0098] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing an image, characterized in that: The processing method comprises: Acquire at least two images to be processed with the target object corresponding to the current moment and each model to be tracked corresponding to the moment before the current moment; For any model to be tracked, determining, among the target objects in the image to be processed, a target object that successfully matches the model to be tracked; Generate a current model according to a target object that has not been successfully matched with any of the to-be-tracked models corresponding to each previous moment; Determine the matching result between the current model and the historical model; wherein the historical model is the model to be tracked within a preset time period before the current moment; Generate an image processing result according to the matching result; The step of determining, for any model to be tracked, a target object in the image to be processed that successfully matches any model to be tracked includes: For any model to be tracked, target objects in different images to be processed are combined to obtain a first object group corresponding to any model to be tracked; Screening the first object groups, and generating a three-dimensional model corresponding to each screened first object group; Determining, among the three-dimensional models corresponding to the screened first object group, a first model that successfully matches any of the models to be tracked; The target object corresponding to the first model is taken as the target object that successfully matches any of the models to be tracked.

2. The processing method according to claim 1, characterized in that: The generating of the current model according to the target object that has not been successfully matched with the to-be-tracked model corresponding to each previous moment includes: Determining a reference image and a non-reference image in the image to be processed; The target object in the reference image that has not been successfully matched with the to-be-tracked model corresponding to each previous moment is used as the to-be-matched object, and the target object in the non-reference image that has not been successfully matched with the to-be-tracked model corresponding to each previous moment is used as the first object; For any object to be matched, first objects in different non-reference images are combined to obtain a second object group corresponding to the object to be matched; Screening the second object group, and generating a three-dimensional model corresponding to each screened second object group according to any of the to-be-matched objects and the second object group; In the three-dimensional models corresponding to the screened second object group, a current model that successfully matches any of the objects to be matched is determined.

3. The processing method according to claim 1, characterized in that: The step of combining target objects in different images to be processed to obtain a first object group corresponding to any model to be tracked includes: For any image to be processed, determining a target object in any image to be processed that is not successfully matched with the model to be tracked as a first unmatched object, and determining whether the first unmatched object is a second object based on a distance between the first unmatched object and any model to be tracked; The second objects in different to-be-processed images are combined to obtain a first object group corresponding to any of the to-be-tracked models.

4. The processing method according to claim 3, characterized in that: The second object is a physical object or a non-physical object; wherein the non-physical object is used to represent an object not captured in the image to be processed, and the physical object is used to represent a target object actually captured in the image to be processed; and the screening of the first object group includes: screening a first object group having at least two physical objects as the screened first object group.

5. The processing method according to claim 2, characterized in that: The combining of the first objects in different non-reference images to obtain a second object group corresponding to any object to be matched includes: For any non-reference image, determining a first object in any non-reference image that is not successfully matched with the object to be matched as a second unmatched object, and determining whether the second unmatched object is a third object according to a distance between the second unmatched object and any object to be matched; The third objects in different non-reference images are combined to obtain a second object group corresponding to any object to be matched.

6. The processing method according to claim 5, characterized in that: The third object is a physical object or a non-physical object; The screening of the second object group includes: screening the second object group having at least two entity objects as the screened second object group.

7. The processing method according to claim 1, characterized in that: Generating an image processing result according to the matching result includes: The model in the current model that successfully matches the historical model and the first model are used as the model to be tracked corresponding to the current moment; wherein the model to be tracked corresponding to the current moment is used to match the target object corresponding to the next moment of the current moment.

8. The processing method according to claim 7, characterized in that: Generating an image processing result according to the matching result includes: When it is determined that both the current model and the historical model fail to match and the preset conditions are met, it is determined that the model to be tracked corresponding to the current moment includes the current model, and an image processing result is generated; wherein the preset conditions include at least one of the following: the current model meets a preset ratio, the number of models to be tracked corresponding to the current moment is less than or equal to a preset number, and the distance between the current model and any model to be tracked corresponding to the current moment is greater than a preset distance.

9. The processing method according to claim 7 or 8, characterized in that: Generating an image processing result according to the matching result also includes: For any model to be tracked corresponding to the current moment, optimizing the model to be tracked corresponding to the current moment according to a model to be tracked corresponding to a previous moment or a historical model that successfully matches the model to be tracked corresponding to the current moment; An image processing result is generated according to the matching conditions between the models to be tracked corresponding to different moments and each model to be tracked corresponding to the current moment after optimization.

10. The processing method according to claim 1, characterized in that: The step of obtaining at least two to-be-processed images with the target object corresponding to the current moment includes: Obtain at least two initial images corresponding to the current moment; For any initial image, determining all initial objects in the initial image; Determining whether any initial object is a target object according to a position of any initial object in any initial image; An initial image with a target object is used as the image to be processed.

11. The processing method according to claim 10, characterized in that: The initial image includes a partial area of ​​the sports field; the initial object includes at least one of an on-field athlete object, a substitute athlete object, and a spectator object; the target object is an on-field athlete object; Determining whether any initial object is a target object based on the position of any initial object in any initial image includes: determining whether any initial object is an on-field athlete object based on the position of a foot key point of any initial object in any initial image projected onto the sports field.

12. An image processing device, characterized in that: The processing device comprises: A data acquisition module, used to acquire at least two images to be processed with the target object corresponding to the current moment and each model to be tracked corresponding to the moment before the current moment; A target object determination module, for determining, for any model to be tracked, a target object that successfully matches any model to be tracked among the target objects in the image to be processed; A current model generation module, used to generate a current model according to a target object that has not been successfully matched with any to-be-tracked model corresponding to each previous moment; A matching result determination module, used to determine the matching result between the current model and the historical model; wherein the historical model is a model to be tracked within a preset time period before the current moment; An image processing result generating module, used for generating an image processing result according to the matching result; The method of determining, for any model to be tracked, a target object that successfully matches any model to be tracked among target objects in the image to be processed, comprises: for any model to be tracked, combining target objects in different images to be processed to obtain a first object group corresponding to any model to be tracked; screening the first object group and generating a three-dimensional model corresponding to each screened first object group; determining, among the three-dimensional models corresponding to the screened first object group, a first model that successfully matches any model to be tracked; and using the target object corresponding to the first model as the target object that successfully matches any model to be tracked.

13. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the processing method described in any one of claims 1 to 11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the processing method described in any one of claims 1 to 11 is implemented.

Images