Target moving object recognition method, device, electronic device and storage medium

Abstract

The present application discloses a target moving object recognition method, device, electronic device and storage medium, and relates to the field of image recognition technology. The method comprises: obtaining a target video frame; performing motion detection on the target video frame to obtain a moving object in the target video frame; screening the moving object to be recognized from the moving object based on the intra-frame features of the moving object; and recognizing the target moving object from the moving object to be recognized based on the recognition rules corresponding to the number of the moving objects to be recognized. The present application can improve the accuracy and efficiency of target moving object recognition.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and more specifically, to a target moving object recognition method, device, electronic device and storage medium. Background Art

[0002] In some ball games, in order to obtain the conditions of each part of the stadium, it is usually hoped to capture the overall image of the stadium. However, the stadiums of some ball games are large, and in the captured overall image of the stadium, the match ball occupies a small proportion of the screen. Online viewers may not be able to identify the match ball well, thereby reducing the audience's online viewing experience.

[0003] In this case, it is usually necessary to identify the match ball in the video to assist online viewers in watching the game. However, using traditional image feature recognition methods to identify the match ball in the video has the problem of low recognition accuracy. Summary of the invention

[0004] In view of this, the embodiments of the present application propose a target moving object recognition method, device, electronic device and storage medium to improve the above-mentioned problems.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying a target moving object, the method comprising: acquiring a target video frame; performing motion detection on the target video frame to obtain a moving object in the target video frame; screening a moving object to be identified from the moving objects based on intra-frame features of the moving object; and identifying a target moving object from the moving objects to be identified based on an identification rule corresponding to the number of moving objects to be identified.

[0006] In the second aspect, the embodiment of the present application provides a target motion object recognition device, which includes: a target video frame acquisition module, a motion detection module, a screening module and a recognition module. Among them, the target video frame acquisition module is used to acquire the target video frame; the motion detection module is used to perform motion detection on the target video frame to obtain the motion object in the target video frame; the screening module is used to filter the motion object to be recognized from the motion object based on the intra-frame features of the motion object; the recognition module is used to recognize the target motion object from the motion object to be recognized based on the recognition rules corresponding to the number of motion objects to be recognized.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is executed by a processor.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above method.

[0010] The embodiments of the present application provide a method, device, electronic device and storage medium for identifying a target moving object. After acquiring a target video frame, the target video frame is first subjected to motion detection to obtain a moving object in the target video frame, and then based on the intra-frame features of the moving object, the moving object to be identified is screened out from the moving object, and finally based on the identification rules corresponding to the number of moving objects to be identified, the target moving object is identified from the moving objects to be identified. Since only the moving objects to be identified screened out from the moving objects are identified, it is equivalent to a preliminary screening of the moving objects, which can exclude interference from other moving objects to a certain extent, thereby improving the accuracy of identifying the target moving object. At the same time, since only the moving objects to be identified screened out from the moving objects are identified, the number of identifications using preset identification rules is reduced, thereby improving the recognition efficiency of the target moving object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A flow chart of a target moving object recognition method proposed in an embodiment of the present application is shown;

[0013] Figure 2 Shows Figure 1 A flowchart of an implementation of S110 in a target moving object recognition method proposed in the illustrated embodiment;

[0014] Figure 3 Shows Figure 1 A flowchart of an implementation of S120 in a target moving object recognition method proposed in the illustrated embodiment;

[0015] Figure 4A schematic diagram of a target video frame in an embodiment of the present application is shown;

[0016] Figure 5 A flowchart of another target moving object recognition method proposed in an embodiment of the present application is shown;

[0017] Figure 6 Shows Figure 5 A schematic diagram of a flow chart of S230 in a target moving object recognition method proposed in the illustrated embodiment;

[0018] Figure 7 Shows Figure 5 A schematic diagram of a flow chart of S230 in another target moving object recognition method proposed in the illustrated embodiment;

[0019] Figure 8 Shows Figure 5 A schematic diagram of a flow chart of S230 in another target moving object recognition method proposed in the illustrated embodiment;

[0020] Fig. 9 A flowchart of another target moving object recognition method proposed in an embodiment of the present application is shown;

[0021] Fig.10 A block diagram of a target moving object recognition device proposed in an embodiment of the present application is shown;

[0022] Fig.11 A structural block diagram of an electronic device for executing a target moving object recognition method according to an embodiment of the present application is shown;

[0023] Fig.12 A storage unit for storing or carrying program codes for implementing a target moving object recognition method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In large-scale ball games such as football and basketball, in order to present a global image of the stadium to online viewers so that they can understand information such as the tactical distribution of the game through the global image, an ultra-wide-angle camera or a panoramic camera can be used to capture the global image of the stadium. The global image of the stadium refers to an image covering the entire stadium area.

[0026] However, since the stadiums used for large-scale ball games are generally large, and the match balls are relatively small compared to the stadiums, the match balls occupy a small proportion of the screen in the collected global images of the stadiums. Online viewers may not be able to identify the match balls well in the video images, thereby reducing the audience's online viewing experience.

[0027] In this case, it is usually necessary to identify the match ball in the video image to assist online viewers in watching the game. For example, the match ball can be identified first, and then the identified match ball can be marked so that online viewers can quickly find the match ball and improve the viewing experience. However, the method of identifying the match ball in the video image in the related art has the problem of low recognition accuracy.

[0028] For example, in some methods, the game ball in the video screen can be identified by feature value recognition. This method directly extracts the feature value of the entire video frame, and then identifies it based on the feature value. Since the identification is based on the feature value of the entire video frame, there is interference from many similar features, so there is a problem of low recognition accuracy.

[0029] In other ways, it can be recognized through neural network deep learning. However, the recognition method of neural network deep learning has certain requirements for the size of the object to be recognized and the resolution of the input video frame. For example, the object to be recognized cannot be less than 20*20 resolution, and the input video frame resolution cannot be greater than 2K. In order to achieve the display effect, the video frame resolution is much larger than the resolution requirement of neural network deep learning. This requires the resolution of the video frame to be scaled to reduce the video frame resolution to meet the requirements of deep learning for the input source size. As a result, the football, which was already very small in the video frame, becomes even smaller due to scaling, and the recognition accuracy of the football by deep learning becomes lower.

[0030] Therefore, the inventors proposed the target motion object recognition method, device, electronic device and storage medium provided in the present application. In the method, after obtaining the target video frame, motion detection is first performed on the target video frame to obtain the motion object in the target video frame, and then based on the intra-frame features of the motion object, the motion object to be identified is screened from the motion objects, and finally based on the recognition rules corresponding to the number of motion objects to be identified, the target motion object is identified from the motion objects to be identified.

[0031] In the aforementioned method, since only the moving objects to be identified that have been screened out from the moving objects are identified, it is equivalent to a preliminary screening of the moving objects, which can eliminate the interference of other moving objects to a certain extent, thereby improving the accuracy of target moving object identification. At the same time, since only the moving objects to be identified that have been screened out from the moving objects are identified, the number of moving objects identified using preset identification rules is reduced, thereby improving the recognition efficiency.

[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] See also Figure 1 , Figure 1 The figure is a flow chart of a target moving object recognition method proposed in an embodiment of the present application, the method comprising the following steps:

[0034] S110, obtaining a target video frame.

[0035] In the process of video image acquisition, the video image is acquired in the form of video image frames (i.e., video frames). Therefore, the video image is composed of multiple video frames. Among them, the target video frame is the video frame in the video image that is subsequently used for motion detection. The video image is a global video image of the stadium of a large ball game event acquired using an ultra-wide-angle camera or a panoramic camera.

[0036] Optionally, the captured global video image may be a real-time video image or a video image captured in advance.

[0037] Among them, it can be understood that the more video frames presented to the audience per unit time, that is, the higher the video frame rate, the smoother the audience can watch. Therefore, in some embodiments, in order to improve the display effect of the target moving object in the video image, all video frames in the video image can be used as target video frames, that is, obtaining the target video frame is to obtain all video frames in the video image, so as to determine the target moving object from all video frames, so that the target moving object can be identified in each video frame in the video image, so that the number of video frames in which the target moving object is identified in the video image finally output to the online audience is more, and the target moving object watched by the audience is smoother.

[0038] The target sports object refers to the sports object that the online audience is paying attention to. For example, in ball games, users are paying attention to the game balls, that is, ball objects, such as football, basketball, volleyball, etc.

[0039] However, considering the device performance limitation, or in order to save the device performance, in other embodiments, the target video frame can also be obtained by obtaining a portion of the video frame in the video image, and the obtained portion of the video frame is used as the target video frame. Figure 2As shown, obtaining the target video frame may specifically include the following steps:

[0040] S111, obtaining a target video image and a video frame extraction frame rate.

[0041] The video frame extraction frame rate may be understood as the frame rate of a video image composed of target video frames extracted from a target video image. In this embodiment, the video frame extraction frame rate may be preset as required.

[0042] S112, extracting a target video frame from the target video image based on the video frame extraction frame rate.

[0043] In order to improve the display effect of the target moving object in the video image, as an implementation method, based on the video frame extraction frame rate, the target video frame can be extracted from the target video source uniformly, that is, the target video frame is extracted at a certain frame number interval. Specifically, the interval frame number can be determined according to the frame rate of the target video image and the video frame extraction frame rate.

[0044] Exemplarily, assuming that the frame rate of the target video image is 60 frames / second and the extracted frame rate of the video image is 30 frames / second, at this time, the interval frame number can be determined to be 1 frame, that is, 1 frame of video frame is extracted as the target video frame every 1 frame of video frame interval.

[0045] As another implementation, based on the video frame extraction frame rate, the target video frame can be extracted from the target video source by randomly extracting the target video frame from the target video image. Still taking the video image extraction frame rate as 30 frames / second as an example, at this time, 30 video image frames can be randomly extracted from the target video image per second as the target video frame.

[0046] S120, performing motion detection on the target video frame to obtain a moving object in the target video frame.

[0047] Moving objects refer to all objects that can move in a video image. For example, in a ball game, moving objects may include a game ball, a player, or other objects that may move.

[0048] After research, it was found that in ball games, the background of the field and the like do not move, but the game ball, athletes, etc., which move. Therefore, in order to reduce the difficulty of subsequent recognition and the amount of recognition data, we can first perform motion detection on the target video frame to obtain the moving object in the target video frame, thereby excluding objects such as the field background, greatly improving the accuracy and efficiency of target moving object recognition.

[0049] Therefore, performing motion detection on the target video frame can be understood as detecting the moving object in the target video frame. There are multiple ways to perform motion detection on the target video frame. Optionally, the target video frame can be subjected to motion detection through a Gaussian mixture model. Optionally, the target video frame can be subjected to motion detection through inter-frame differences.

[0050] In addition, considering that when using ultra-wide-angle cameras or panoramic cameras and other video equipment to capture the global image of the stadium, it is inevitable that objects outside the stadium will be captured, such as spectators outside the stadium, cleaning staff, or players using balls to warm up, and these objects can also move. If these objects are also used for subsequent identification of target moving objects, the recognition accuracy and efficiency will be reduced.

[0051] Therefore, in order to avoid the influence of the motion objects outside the stadium on the recognition accuracy and recognition efficiency, as an implementation method, Figure 3 As shown, performing motion detection on a target video frame to obtain a moving object in the target video frame may specifically include the following steps:

[0052] S121, obtaining a picture of a valid motion area in a target video frame.

[0053] The effective movement area can be understood as the area within the possible movement range of the target moving object. For example, in ball games, the game ball may usually move within the stadium or within a certain range of the stadium boundary. Therefore, the effective movement area can refer to the area within the stadium or within a certain range of the stadium boundary.

[0054] Considering that when an ultra-wide-angle camera or a panoramic camera is used to capture the global image of the stadium, the ultra-wide-angle camera or the panoramic camera usually does not change the capture position, so the scene area of ​​the captured video image does not change. Therefore, the effective motion area in the acquired target video frame can be calibrated in advance to obtain the image of the effective area. For example, the boundary of the stadium is calibrated in the video image in advance, or a certain range outside the boundary of the stadium is calibrated.

[0055] Optionally, a polygon may be used to enclose the range of the playing field or a range within a certain range of the playing field boundary, so as to obtain the image within the polygon as the image of the effective motion area.

[0056] For example, reference Figure 4 , Figure 4 A schematic diagram of a target video frame is shown. Figure 4 The target video frame shown in shows both the scene inside the stadium and the scene outside the stadium. At this time, the stadium area can be circled with a polygon to obtain the scene of the effective motion area in the target video frame.

[0057] S122, performing motion detection on the picture in the effective motion area to obtain the moving object in the picture in the effective motion area.

[0058] By performing motion detection on the images in the effective motion area, the influence of spectators, cleaning staff, or players using balls for warm-up on the target moving object is eliminated, further improving the accuracy and efficiency of identifying the target moving object.

[0059] S130, based on the intra-frame features of the moving objects, screen the moving objects to be identified from the moving objects.

[0060] The moving objects to be identified are the moving objects that are subsequently identified using preset identification rules. It is understandable that even the moving objects obtained from the effective motion area screen still include all the objects that have moved, such as athletes, game balls, flags waved by referees, or other objects that may move. The number of these objects may still be large. If all these moving objects are directly identified, there is still a problem of low recognition efficiency. Therefore, in order to further improve the recognition efficiency, as an implementation method, after obtaining the moving objects, the moving objects can be further screened based on the intra-frame features of the moving objects to obtain the moving objects to be identified, thereby further reducing the moving objects that are actually used for identification in the future.

[0061] Among them, the intra-frame feature refers to the feature obtained from a video frame.

[0062] S140: Based on the recognition rule corresponding to the number of the moving objects to be recognized, a target moving object is recognized from the moving objects to be recognized.

[0063] It is understandable that after the moving objects are screened in step S130 to obtain the moving objects to be identified, the number of moving objects to be identified is reduced again. At this time, the number of moving objects to be identified may be one or more. At this time, different recognition rules can be directly selected based on the number of moving objects to be identified to identify the small number of moving objects to be identified, thereby obtaining the target moving object.

[0064] In addition, considering that under normal circumstances, there is a game ball in the stadium by default, therefore, if the number of the to-be-identified moving objects is one, the one to-be-identified moving object can be directly determined as the target moving object. In this case, based on the recognition rule corresponding to the number of the to-be-identified moving objects, the target moving object is identified from the to-be-identified moving objects, including: when the number of the to-be-identified moving objects screened from the moving objects is one, the to-be-identified moving object is determined as the target moving object.

[0065] In addition, considering that under normal circumstances, in addition to the game ball, there may be athletes waiting to identify moving objects in the stadium. Similarly, because under normal circumstances, there is a game ball by default in the stadium, therefore, when the number of moving objects to be identified is greater than one, the moving object to be identified cannot be directly determined as the target moving object, and each moving object to be identified needs to be identified separately. In this case, based on the recognition rules corresponding to the number of moving objects to be identified, the target moving object is identified from the moving objects to be identified, including: when the number of moving objects to be identified screened from the moving objects is greater than one, the images corresponding to each moving object to be identified are input into the object classifier, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

[0066] In this embodiment, since the moving object to be identified is a partial picture in the target video frame, resolution scaling is not performed, and the resolution can meet the requirements of the neural network model for input data. Therefore, the object classifier can be a trained neural network model. The neural network model is used to identify the moving object to be identified, thereby improving the recognition accuracy.

[0067] The object classifier may be a supervised model, that is, trained using sample motion objects with classification labels. The sample motion objects may be obtained by the above steps S110-S130, that is, first obtaining a sample video frame, then performing motion detection on the sample video frame to obtain the motion objects in the sample video frame, and then screening the sample motion objects from the motion objects to obtain the sample motion objects. After obtaining the sample motion objects, the sample motion objects with classification labels may be obtained by manual annotation.

[0068] Optionally, the neural network model can adopt open source models such as TensorFlow (a symbolic mathematical system based on data flow programming, which is widely used in the programming implementation of various machine learning algorithms) and Caffe (Convolutional Architecture for Fast Feature Embedding).

[0069] In some embodiments, the object classifier can be used to receive a moving object to be detected, and then output whether the moving object to be detected is a target moving object. For example, if a moving object of a "human head" is input into the object classifier, and the target moving object is a football, then the output result of the object classifier is negative, that is, the moving object is not the target moving object. In this way, the target moving object can be identified from multiple moving objects to be identified.

[0070] In other embodiments, the object classifier can be used to receive a moving object to be detected, and then output the object type of the moving object to be detected, so as to determine the target moving object from the output object type. For example, if a moving object of a "human head" is input into the object classifier, the output result of the object classifier is that the moving object is a "human head", and if a moving object of a "soccer ball" is input into the object classifier, the output result of the object classifier is that the moving object is a "soccer ball", so that the type of each moving object to be identified can be obtained, and thus the selected target moving object can be identified from multiple moving objects to be identified.

[0071] Since the output result of the object classifier can be a target moving object or a moving object of other classifications, not only the target moving object can be obtained, but also the classification of other moving objects can be obtained, for example, the classification of athletes can be obtained. Therefore, when online viewers want to pay attention to the position of the athlete, the position of the athlete can also be framed and displayed in the video image.

[0072] The present application provides a method for identifying a target moving object. After acquiring a target video frame, the target video frame is firstly subjected to motion detection to obtain a moving object in the target video frame, and then based on the intra-frame features of the moving object, the moving object to be identified is screened out from the moving object, and finally based on the identification rule corresponding to the number of the moving objects to be identified, the target moving object is identified from the moving objects to be identified. Since only the moving objects to be identified screened out from the moving objects are identified, it is equivalent to a preliminary screening of the moving objects, which can exclude interference from other moving objects to a certain extent, thereby improving the accuracy of identifying the target moving object. At the same time, since only the moving objects to be identified screened out from the moving objects are identified, the number of identifications using preset identification rules is reduced, thereby improving the recognition efficiency of the target moving object.

[0073] See also Figure 5 , Figure 5 FIG. 1 is a flow chart of a method for identifying a moving target object according to another embodiment of the present application. The method may include the following steps:

[0074] S210, obtaining a target video frame.

[0075] S220, performing motion detection on the target video frame to obtain a moving object in the target video frame.

[0076] In some implementations, in order to completely select the moving object and facilitate determination of the position and actual size of the moving object, a rectangular frame may be used to select the moving object obtained by motion detection. Figure 4 As shown, various moving objects are selected by rectangular boxes.

[0077] S230 , based on the intra-frame features of the moving objects, screen the moving objects to be identified from the moving objects.

[0078] There are multiple types of intra-frame features of moving objects.

[0079] In some embodiments, the intra-frame feature may be the actual size of the current position of the moving object in the target video frame. Figure 6 As shown, based on the intra-frame features of the moving objects, the moving objects to be identified are screened from the moving objects, which may specifically include the following steps:

[0080] S231A, obtaining predicted sizes of the target moving object at different positions in the target video frame, and the actual size of the moving object at the current position in the target video frame.

[0081] It is understandable that in the global image of the stadium captured by an ultra-wide-angle camera or a panoramic camera, each object shows a pattern of being larger near and smaller far away. That is to say, in the actual scene, the display size of the moving object in the final captured video image is different due to the different distances from the acquisition device. Based on this discovery, when the position of the acquisition device such as an ultra-wide-angle camera or a panoramic camera is determined, the size of the target moving object corresponding to different positions in the target video frame can be calibrated in advance. The pre-calibrated size is the predicted size of the target moving object at different positions in the target video frame.

[0082] It is understandable that the moving object is at a position in the target video frame, that is, the current position of the moving object, and similarly, the moving object also has an actual size in the target video frame. Therefore, the actual size of the current position of the moving object in the target video frame can be directly obtained from the target video frame.

[0083] After the moving object is selected by using a rectangular frame, the actual size of the moving object can be obtained by multiplying the resolution length and width of the rectangular frame.

[0084] S232A: When the actual size of the moving object at the current position matches the predicted size of the current position, determine that the moving object is a moving object to be identified.

[0085] In some implementations, the predicted size may be a range, in which case the actual size of the moving object at the current position matches the predicted size at the current position means that the actual size is within the range of the predicted size.

[0086] Therefore, when the actual size of the moving object at its current position matches the predicted size of its current position, it can be determined that the moving object meets the requirements in terms of size, and moving objects whose sizes do not match the target moving object can be preliminarily screened out, such as flags waved by referees, etc., so that the moving objects that meet the requirements can be determined as the moving objects to be identified.

[0087] It can be seen that this embodiment can reduce the number of moving objects that are subsequently identified using recognition rules, eliminate the interference of interfering moving objects, and improve the recognition accuracy and recognition efficiency of target moving objects.

[0088] In other embodiments, the intra-frame feature may be the actual aspect ratio of the moving object in the target video frame. Figure 7 As shown, based on the intra-frame features of the moving objects, the moving objects to be identified are screened from the moving objects, which may specifically include the following steps:

[0089] S231B, obtaining a predicted aspect ratio of the target moving object in the target video frame, and an actual aspect ratio of the moving object in the target video frame.

[0090] It can be understood that the predicted aspect ratio of the target moving object in the target video frame refers to the aspect ratio that the target moving object should be displayed in the target video frame. For example, for ball objects such as footballs and basketballs, they should be displayed in the target video frame with an aspect ratio close to a square, that is, the aspect ratio is close to one to one. For athletes, they should be displayed with an aspect ratio of a long strip.

[0091] The actual aspect ratio of the moving object in the target video frame can be directly obtained from the target video frame.

[0092] After the moving object is selected by using a rectangular frame, the actual aspect ratio of the moving object can be obtained by the ratio of the resolution length and width of the rectangular frame.

[0093] S232B: When the actual aspect ratio of the moving object matches the predicted aspect ratio of the target moving object, determine that the moving object is a moving object to be identified.

[0094] In some implementations, the predicted aspect ratio may be a range, and in this case, the actual aspect ratio of the moving object matches the predicted aspect ratio of the target moving object means that the actual aspect ratio is within the range of the predicted aspect ratio.

[0095] Therefore, when the actual aspect ratio of the moving object matches the predicted aspect ratio of the target moving object, it can be determined that the moving object meets the requirements in terms of aspect ratio. Similarly, moving objects whose aspect ratios do not match those of the target moving object can be preliminarily screened out, thereby determining the moving objects that meet the requirements as the moving objects to be identified, further reducing the number of moving objects that are subsequently identified using recognition rules, and improving the recognition accuracy and efficiency of the target moving objects.

[0096] For example, when an athlete's lower limbs do not move and only his arms move, the moving object is the athlete's arms, and the actual length-to-width ratio of the arms is close to the ratio of a long strip, while when the target moving object is a football, the predicted length-to-width ratio of the football is close to the ratio of a square. Therefore, the two do not match, and moving objects such as "arms" can be filtered out.

[0097] In other embodiments, some moving objects are symmetrical, such as a game ball, a complete body of an athlete, etc., while some moving objects are not symmetrical. For example, for an athlete whose lower limbs are immobile and only one arm moves, the single "arm" moving object identified at this time is not symmetrical. Based on the symmetry of the moving object, in some embodiments, the intra-frame feature can be a color distribution parameter of the moving object. In this case, Figure 8 As shown, based on the intra-frame features of the moving objects, the moving objects to be identified are screened from the moving objects, which may specifically include the following steps:

[0098] S231C, obtaining color distribution parameters of the moving object.

[0099] The color distribution parameter of the moving object refers to the color distribution of the moving object, and optionally, may be the RGB color distribution of the moving object in the target video frame.

[0100] After the moving object is selected by using a rectangular frame, the color distribution parameter of the moving object may be a color distribution parameter of the portion selected by the rectangular frame.

[0101] S232C, performing symmetry detection on the moving object based on the color distribution parameter to obtain a symmetry detection result of the moving object.

[0102] It is understandable that for a moving object with symmetry, its color distribution has certain regularities, for example, a symmetrical color distribution along the axis of symmetry. Therefore, the moving object can be tested for symmetry based on the color distribution parameters to obtain the symmetry detection result of the moving object.

[0103] In some embodiments, after the moving object is selected by using a rectangular frame, the rectangular frame containing the moving object may be divided, for example, into four grids, nine grids, or sixteen grids, etc. Taking the nine grids as an example, the four corners, i.e., the upper left, upper right, lower left, and lower right grids, are each subjected to RGB color comparison to calculate similarity, or the upper left grid is selected to perform RGB color comparison with the upper right grid to calculate similarity, or the lower left grid is selected to perform RGB color comparison with the lower right grid to calculate similarity.

[0104] It should be noted that the similarity calculation method is not limited in the embodiments of the present application. For example, the RGB color comparison can be performed on the entirety of the three left grids in the nine-grid and the entirety of the three right grids to calculate a similarity. For another example, the RGB color comparison can be performed on the upper left, upper right, lower left, and lower right grids of the nine-grid in turn to calculate multiple similarities.

[0105] After calculating the similarity, the calculated similarity (which can be one or more) can be compared with a similarity threshold. If the calculated similarity is greater than the similarity threshold, it can be determined that the symmetry detection result of the moving object is symmetrical. If the calculated similarity is less than or equal to the similarity threshold, it can be determined that the symmetry detection result of the moving object is not symmetrical.

[0106] S233C, when the symmetry detection result matches the predicted symmetry of the target moving object, determine that the moving object is a moving object to be identified.

[0107] It is understandable that the predicted symmetry of the target moving object may be symmetric or non-symmetric. For example, when the game ball is selected as the target moving object, the predicted symmetry of the target moving object is symmetric. However, some moving objects do not have symmetry. For example, for an athlete whose lower limbs are immobile and only one arm moves, the single "arm" moving object identified at this time is non-symmetric. When a non-symmetric moving object is selected as the target moving object, the predicted symmetry of the target moving object is non-symmetric.

[0108] Therefore, when the predicted symmetry of the target moving object is symmetrical, the moving object with symmetrical results can be determined as the object to be identified, and when the predicted symmetry of the target moving object is not symmetrical, the moving object with asymmetric results can be determined as the object to be identified.

[0109] In this embodiment, based on the color distribution parameters, a symmetry detection is performed on the moving object to obtain a symmetry detection result of the moving object, and then based on the symmetry detection result that the moving object is symmetrical, it is determined that the moving object is a moving object to be identified. It can be determined that the color distribution parameter of the moving object meets the requirements. Similarly, the moving objects whose color distribution parameters do not meet the requirements of the target moving objects can be preliminarily screened out, so that the moving objects that meet the requirements can be determined as the moving objects to be identified, further reducing the number of moving objects that are subsequently identified using recognition rules, and improving the recognition accuracy and recognition efficiency of the target moving objects.

[0110] It should be noted that when the moving object to be identified is screened from the moving objects based on the intra-frame features of the moving object, each intra-frame feature can be used separately, for example, the actual size of the moving object in the target video frame at the current position in the video frame is used separately, the actual aspect ratio of the moving object in the target video frame is used separately, or the color distribution parameter of the moving object is used separately. Any two or more of the intra-frame features can also be used in combination at the same time. For example, the actual size of the moving object in the target video frame at the current position in the video frame and the color distribution parameter of the moving object are used in combination, or the actual size of the moving object in the target video frame at the current position in the video frame, the actual aspect ratio of the moving object in the target video frame, and the color distribution parameter of the moving object are used in combination at the same time. Moreover, when any two or more of the intra-frame features are used in combination at the same time, the various intra-frame features can be used regardless of the order.

[0111] In this embodiment, there is no specific limitation on the intra-frame features of the moving objects based on which the moving objects to be identified are screened from the moving objects.

[0112] S241: When the number of the to-be-identified moving objects screened from the moving objects is one, determine the to-be-identified moving object as a target moving object.

[0113] S242, when the number of moving objects to be identified screened from the moving objects is greater than one, the images corresponding to the moving objects to be identified are input into the object classifier, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

[0114] The target moving object recognition method of this embodiment, in the process of screening the moving objects to be identified from the moving objects based on the intra-frame features of the moving objects, proposes a variety of specific intra-frame features to screen the moving objects, has a wide range of applications, and reduces the difficulty of screening the moving objects.

[0115] See also Fig. 9 , Fig. 9 FIG. 1 is a flow chart of a target moving object recognition method proposed in another embodiment of the present application. The method is applied to a case where the number of moving objects to be recognized in a target video frame is greater than one. The method may include the following steps:

[0116] S310, obtaining a target video frame.

[0117] S320, performing motion detection on the target video frame to obtain a moving object in the target video frame.

[0118] S330, based on the intra-frame features of the moving objects, screen the moving objects to be identified from the moving objects.

[0119] In this embodiment, the number of moving objects to be identified screened from the moving objects based on the intra-frame features of the moving objects is greater than one.

[0120] S340, obtaining the actual distance between each moving object to be identified and a credible object in an adjacent target video frame corresponding to the same video frame, wherein the credible object is the moving object to be identified corresponding to when the number of the moving objects to be identified screened from the moving objects is one.

[0121] It is understandable that after screening the moving objects in a certain target video frame, it is still possible to obtain multiple moving objects to be detected, however, in some cases, the number of target moving objects is limited. For example, under normal circumstances, there is only one match ball in the stadium by default. Therefore, if all the moving objects to be detected are identified in a random order based on the corresponding recognition rules, the target moving object may be identified only when the last moving object to be detected is identified, which reduces the recognition efficiency of the target moving object. Therefore, in order to further improve the recognition efficiency of the target moving object, the actual distance between each moving object to be identified and the credible object in the adjacent target video frame corresponding to the same video frame can be obtained first.

[0122] In combination with the foregoing content, it can be seen that in some cases, for certain target video frames, based on preset screening rules, the number of moving objects to be identified screened from moving objects may be one. At this time, since there is a target moving object in the default target video frame, the moving object to be detected can be determined as the target moving object, and this target moving object is understood as a trusted object.

[0123] Furthermore, since the time interval between two adjacent target video frames is short, in actual scenes, the target moving object cannot move a long distance within a short time interval, and in the captured two adjacent target video frames, the distance difference between the target moving objects is even smaller. Therefore, after obtaining the credible objects in the adjacent target video frames, for each moving object to be identified in the target video frame currently being identified, the actual distance between each moving object to be identified and the credible object in the adjacent video frame corresponding to the same video frame can be obtained first.

[0124] In actual scenarios, the acquisition angle of the video image acquisition device usually does not change, so the background position distribution in the acquired video frame does not change. In this case, as an implementation method, different target video frames can be mapped into the same target video frame, that is, in different target video frames, the same coordinate system can be established, that is, the coordinate origin is the same in the two video frames, so as to obtain the coordinates of the trusted object and the coordinates of each moving object to be identified. Since the coordinate system is the same, the coordinates in different target video frames can be directly used for distance calculation to obtain the actual distance between each moving object to be identified and the trusted object in the same video frame.

[0125] S350, according to the order of the actual distance between each moving object to be identified and the corresponding credible object in the adjacent video frame on the same video frame from small to large, the image corresponding to each moving object to be identified is input into the object classifier in turn, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

[0126] Since the distance difference between the target moving objects in the two adjacent target video frames obtained is small, after obtaining the actual distances between each moving object to be identified and the credible objects in the adjacent target video frames corresponding to the same video frame, the moving objects to be identified and the credible objects in the adjacent video frames can be sorted in ascending order according to the actual distances between the moving objects to be identified and the credible objects in the adjacent video frames corresponding to the same video frame. It can be understood that the smaller the actual distance, the greater the probability that the moving object to be identified is the target moving object. Therefore, the images corresponding to each moving object to be identified can be input into the object classifier for identification in order of the actual distance from small to large, until the target moving object is identified from the moving objects to be identified by the object classifier.

[0127] By adopting the method of this embodiment, the recognition can start from the most likely target moving object, which can improve the recognition efficiency of the target moving object.

[0128] It should be noted that the present application provides some specific examples of implementation methods above. Under the premise of not contradicting each other, the various embodiment examples can be arbitrarily combined to form a new target moving object recognition method. It should be understood that the new target moving object recognition method formed by combining any examples should fall within the protection scope of the present application.

[0129] See also Fig.10 , Fig.10 A block diagram of a target moving object recognition device 400 proposed in an embodiment of the present application is shown. The device 400 may include: a target video frame acquisition module 410, a motion detection module 420, a screening module 430 and a recognition module 440.

[0130] A target video frame acquisition module 410 is used to acquire a target video frame;

[0131] A motion detection module 420 is used to perform motion detection on a target video frame to obtain a moving object in the target video frame;

[0132] A screening module 430 is used to screen the moving objects to be identified from the moving objects based on the intra-frame features of the moving objects;

[0133] The identification module 440 is used to identify the target moving object from the moving objects to be identified based on the identification rule corresponding to the number of moving objects to be identified.

[0134] As an implementation manner, the target video frame acquisition module 410 is further used to acquire a target video image and a video frame extraction frame rate; and extract a target video frame from the target video image based on the video frame extraction frame rate.

[0135] As an implementation manner, the motion detection module 420 is further used to obtain a picture of a valid motion area in the target video frame; perform motion detection on the picture of the valid motion area to obtain a moving object in the picture of the valid motion area.

[0136] As an implementation mode, the screening module 430 is also used to obtain the predicted sizes of the target moving object at different positions in the target video frame, and the actual size of the moving object at the current position in the target video frame; when the actual size of the moving object at the current position matches the predicted size at the current position, the moving object is determined to be the moving object to be identified.

[0137] As an implementation mode, the screening module 430 is also used to obtain the predicted aspect ratio of the target moving object in the target video frame, and the actual aspect ratio of the moving object in the target video frame; when the actual aspect ratio of the moving object matches the predicted aspect ratio of the target moving object, the moving object is determined to be the moving object to be identified.

[0138] As an implementation mode, the screening module 430 is also used to obtain the color distribution parameters of the moving object; based on the color distribution parameters, perform symmetry detection on the moving object to obtain a symmetry detection result of the moving object; when the symmetry detection result matches the predicted symmetry of the target moving object, determine that the moving object is a moving object to be identified.

[0139] As an embodiment, the recognition module 440 is also used to determine the moving object to be identified as the target moving object when the number of moving objects to be identified screened from the moving objects is one; or, when the number of moving objects to be identified screened from the moving objects is greater than one, input the images corresponding to each moving object to be identified into the object classifier, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

[0140] As an implementation mode, the number of moving objects to be identified is greater than one. In this case, the identification module 440 is further used to obtain the actual distance between each moving object to be identified and a credible object in an adjacent video frame corresponding to the same video frame, where the credible object is the moving object to be identified when the number of moving objects to be identified screened from the moving objects is one; and according to the order of the actual distance between each moving object to be identified and the credible object in the adjacent video frame corresponding to the same video frame from small to large, the images corresponding to each moving object to be identified are input into the object classifier in sequence, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

[0141] The present application provides a target moving object recognition device, which only recognizes the moving objects to be recognized that are screened out from the moving objects, which is equivalent to a preliminary screening of the moving objects. It can eliminate the interference of other moving objects to a certain extent, thereby improving the accuracy of target moving object recognition. At the same time, since only the moving objects to be recognized that are screened out from the moving objects are recognized, the number of recognitions using preset recognition rules is reduced, thereby improving the recognition efficiency of the target moving object.

[0142] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0143] The following will be combined Fig.11 An electronic device provided by the present application is described.

[0144] See also Fig.11 Based on the above target moving object recognition method, the embodiment of the present application also provides another electronic device 200 including a processor 104 that can execute the above target moving object recognition method. The electronic device 200 can be a smart phone, a tablet computer, a computer or a portable computer. The electronic device 200 also includes a memory 104, a network module 106 and a screen 108. The memory 104 stores a program that can execute the content of the above embodiment, and the processor 102 can execute the program stored in the memory 104.

[0145] Among them, the processor 102 may include one or more cores for processing data and a message matrix unit. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes various functions and processes data of the electronic device 200 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented separately through a communication chip.

[0146] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the terminal 100 during use.

[0147] The network module 106 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, for example, communicate with an audio playback device. The network module 106 may include various existing circuit elements for performing these functions, for example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a user identity module (SIM) card, a memory, etc. The network module 106 may communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other devices via a wireless network. The above-mentioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 106 may exchange information with a base station.

[0148] The screen 108 can display interface content and can also be used to respond to touch gestures.

[0149] It should be noted that, in order to achieve more functions, the electronic device 200 can also protect more devices, for example, it can also protect a structured light sensor for collecting facial information or a camera for collecting irises.

[0150] Please refer to Fig.12 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1100 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0151] The computer readable storage medium 1100 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1100 has storage space for program code 1110 that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 1110 can be compressed, for example, in an appropriate form.

[0152] Based on the above target moving object recognition method, according to one aspect of the embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementations.

[0153] In summary, the target motion object recognition method, device, electronic device, storage medium and computer program product or computer program provided by the embodiments of the present application, since only the motion objects to be recognized selected from the motion objects are recognized, it is equivalent to a preliminary screening of the motion objects, which can eliminate the interference of other motion objects to a certain extent, thereby improving the accuracy of target motion object recognition. At the same time, since only the motion objects to be recognized selected from the motion objects are recognized, the number of recognitions using preset recognition rules is reduced, thereby improving the recognition efficiency of the target motion object.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying a moving target object, characterized in that: include: Get the target video frame; Performing motion detection on the target video frame to obtain a moving object in the target video frame; Based on the intra-frame features of the moving objects, screening the moving objects to be identified from the moving objects; Based on the recognition rule corresponding to the number of the moving objects to be recognized, identifying the target moving object from the moving objects to be recognized; The step of selecting the moving object to be identified from the moving objects based on the intra-frame features of the moving objects includes: The predicted sizes of the target moving object at different positions in the target video frame and the actual size of the moving object at the current position in the target video frame are obtained; when the actual size of the moving object at the current position matches the predicted size at the current position, the moving object is determined to be a moving object to be identified, the predicted size is a size range, and the actual size of the moving object at the current position matches the predicted size at the current position, indicating that the actual size is within the range of the predicted size.

2. The method according to claim 1, characterized in that The step of identifying a target moving object from the moving objects to be identified based on the identification rule corresponding to the number of the moving objects to be identified includes: When the number of the to-be-identified moving objects screened from the moving objects is one, determining the to-be-identified moving object as a target moving object; or When the number of moving objects to be identified screened from the moving objects is greater than one, the images corresponding to the moving objects to be identified are input into an object classifier, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

3. The method according to claim 1, characterized in that The number of the moving objects to be identified is greater than one, and the identifying rule corresponding to the number of the moving objects to be identified, identifying the target moving object from the moving objects to be identified, comprises: Acquire the actual distance between each of the moving objects to be identified and the credible objects in the adjacent video frames corresponding to the same video frame, wherein the credible object is the moving object to be identified corresponding to when the number of the moving objects to be identified screened from the moving objects is one; According to the order of the actual distance between each of the moving objects to be identified and the credible objects in the adjacent video frames on the same video frame from small to large, the images corresponding to each of the moving objects to be identified are input into the object classifier in turn, so as to identify the target moving object from the moving objects to be identified through the object classifier, wherein the object classifier is trained by sample moving objects with classification labels determined from sample video frames.

4. The method according to claim 1, characterized in that The step of obtaining a target video frame comprises: Obtain target video image and video frame extraction frame rate; A target video frame is extracted from the target video image based on the video frame extraction frame rate.

5. The method according to claim 1, characterized in that The performing motion detection on the target video frame to obtain the moving object in the target video frame includes: Acquire a picture of a valid motion area in the target video frame; Motion detection is performed on the picture in the effective motion area to obtain the moving object in the picture in the effective motion area.

6. A target moving object recognition device, characterized in that: The device comprises: A target video frame acquisition module, used to acquire a target video frame; A motion detection module, used to perform motion detection on the target video frame to obtain a moving object in the target video frame; A screening module, used for screening the moving objects to be identified from the moving objects based on the intra-frame features of the moving objects; An identification module, configured to identify a target moving object from the moving objects to be identified based on an identification rule corresponding to the number of the moving objects to be identified; Among them, the screening module is specifically used to obtain the predicted sizes of the target moving object at different positions in the target video frame, and the actual size of the moving object at the current position in the target video frame; when the actual size of the moving object at the current position matches the predicted size at the current position, the moving object is determined to be a moving object to be identified, the predicted size is a size range, and the actual size of the moving object at the current position matches the predicted size at the current position, indicating that the actual size is within the range of the predicted size.

7. An electronic device, characterized in that: The method comprises a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 5 is executed.

Images