Event detection method, device, terminal equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence technology, and provides an event detection method, device, terminal device and storage medium. The event detection method comprises: obtaining a video frame to be identified; detecting the image area where the specified event contained in the video frame to be identified is located; inputting the image area into a trained event classification model for processing to obtain the classification result of the specified event; and correcting the classification result of the specified event according to the object detection result of the video frame to be identified. The method provided by the present application can obtain accurate event classification results.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to an event detection method, apparatus, terminal device and storage medium. Background Art

[0002] The development of information technology and artificial intelligence technology has made digitalization and intelligence the mainstream trend of the future. For example, in urban management activities, in order to determine whether there are urban management events in the urban management area, cameras are generally set up in advance to obtain video information through the device head, and the video information is processed through the event detection algorithm to obtain event classification results, so as to manage the city according to the event classification results, thereby achieving the purpose of smart urban management. However, since the current event detection algorithm generally has the problem of low accuracy, the event classification results obtained have errors, which is not conducive to the development of urban management activities.

[0003] Therefore, there is an urgent need to provide a method that can process video information to obtain accurate event classification results. Summary of the invention

[0004] The embodiments of the present application provide an event detection method, apparatus, terminal device and storage medium to solve the problem of how to obtain accurate event classification results.

[0005] In a first aspect, an embodiment of the present application provides an event detection method, including:

[0006] Get the video frame to be recognized;

[0007] Detecting an image region where a designated event contained in the to-be-identified video frame is located;

[0008] Inputting the image region into a trained event classification model for processing to obtain a classification result of the designated event;

[0009] The classification result of the designated event is modified according to the object detection result of the video frame to be identified.

[0010] Compared with the prior art, in the event detection method provided in the embodiment of the present application, a video frame to be identified is obtained, and it is detected whether the video frame to be identified contains an image area where a specified event is located. If the image area where the specified event is located is detected in the video frame to be identified, the image area is input into the trained event classification model for processing to obtain the classification result of the specified event. Furthermore, in order to avoid the classification error of the specified event, the object detection result of the video frame to be identified is used to understand the object situation in the image area where the specified event is located, so as to facilitate the correction of the classification result of the specified event according to the object situation of the image area where the specified event is located, and obtain the correct event classification result corresponding to the image area where the specified event is located.

[0011] In a second aspect, an embodiment of the present application provides an event detection device, including:

[0012] An acquisition module, used for acquiring a video frame to be identified;

[0013] A detection module, used for detecting an image region where a designated event contained in the video frame to be identified is located;

[0014] A classification module, used for inputting the image area into a trained event classification model for processing to obtain a classification result of the specified event;

[0015] The correction module is used to correct the classification result of the specified event according to the object detection result of the video frame to be identified.

[0016] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the event detection method when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the event detection method when executed by a processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed on a terminal device, the terminal device executes the event detection method described in any one of the above-mentioned first aspects.

[0019] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 It is a schematic diagram of the implementation flow of the event detection method provided in one embodiment of the present application.

[0022] Figure 2 It is a schematic diagram of a video frame to be identified provided by an embodiment of the present application.

[0023] Figure 3 It is a schematic diagram of a specific implementation flow of step S14 of the event detection method provided in an embodiment of the present application.

[0024] Figure 4 It is a structural diagram of an event detection device provided in an embodiment of the present application.

[0025] Figure 5 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0027] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0028] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0030] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0031] See also Figure 1 , Figure 1 This is a flow chart of an event detection method provided in an embodiment of the present application. In this embodiment, the event detection method can be applied to terminal devices such as mobile phones, tablet computers, desktop computers, notebooks, PDAs, cloud servers, ultra-mobile personal computers (UMPCs), and netbooks. The present application embodiment does not impose any restrictions on the specific types of terminal devices.

[0032] like Figure 1 As shown, the event detection method provided in the embodiment of the present application includes the following steps:

[0033] S11: Obtain a video frame to be identified.

[0034] As an example of the present application, the video frame to be identified is an image obtained by shooting a target area, so the situation in the target area can be understood through the video frame to provide a data basis for detecting and classifying the information in the video frame to be identified. The target area is a physical area to be understood as required, such as a street monitored by a surveillance camera.

[0035] It can be understood that the situation in the target area can be understood through the video frame to be identified. Figure 2 The obtained video frame to be identified is used to understand the situation of pedestrians and vehicles on the street.

[0036] In an application, the number of the acquired video frames to be identified may be one or more.

[0037] As a possible implementation of this embodiment, a specific implementation of obtaining the video frame to be identified may include:

[0038] Get the target video.

[0039] The target video is sampled according to a specified sampling rate to obtain a video frame to be identified, wherein the specified sampling rate is determined according to a frame rate of the target video.

[0040] In this embodiment, the target video is a video acquired by a camera device, for example, a surveillance video acquired by a camera installed on a street.

[0041] The specified sampling rate is used to describe the number of video frames to be identified extracted from the target video per second. It is understandable that the specified sampling rate can be preset according to actual needs. For example, the frame rate of the target video can be 25fps or 30fps. Setting the specified sampling rate to 5 frames per second means that 5 consecutive video frames are extracted from the target video within one second, and each extracted video frame is used as a video frame to be identified.

[0042] It is understandable that in order to sample the target video quickly, the correspondence between the frame rate of the video and the specified sampling rate is determined in advance. Therefore, when the frame rate of a target video is known, the specified sampling rate can be determined according to the frame rate of the target video, so that the target video can be sampled according to the specified sampling rate to obtain at least one frame of the video to be identified. For example, when the frame rate of a target video is determined to be 25fps, the corresponding specified sampling rate is 5 frames per second, so the target video can be sampled based on the specified sampling rate to obtain the video frame to be identified.

[0043] In some embodiments, a uniform sampling method is used to determine at least one video frame to be identified from each video frame of the target video. The uniform sampling method describes extracting a plurality of consecutive video frames from each video frame of the target video using a set specified sampling rate.

[0044] For example, the sampling rate is specified to be 5 frames per second, that is, 5 consecutive video frames are extracted from each video frame of the target video every 1 second, that is, 5 video frames to be identified are extracted.

[0045] S12: Detecting the image region where the designated event contained in the to-be-identified video frame is located.

[0046] As an example of the present application, the designated event is an event set according to actual needs. For example, the designated event can be a city management event, such as Figure 2 The black boxes correspond to pedestrians and the objects they pull.

[0047] The image region is the region in the video frame to be identified that describes the specified event. For example, Figure 2 The image area corresponding to the black frame.

[0048] In this embodiment, since the video frame to be identified can be used to understand the situation of a target area, in order to determine whether a specified event exists, the video frame to be identified can be detected to determine whether the video frame to be identified contains an image area where the specified event is located.

[0049] In one embodiment, in an application, if there are multiple video frames to be identified, the image region where the designated event is located contained in each video frame to be identified is detected according to the time sequence of acquiring the video frames to be identified.

[0050] In one embodiment of the present application, detecting an image region where a designated event contained in a to-be-identified video frame is located includes:

[0051] The video frame to be identified is input into the trained event detection model for processing to obtain the image area where the specified event is located in the video frame to be identified.

[0052] As an example of the present application, in order to efficiently perform event detection, after obtaining the video frame to be identified, the video frame to be identified is input into a trained event detection model for processing, so as to determine the image area where the specified event is located in the video frame to be identified.

[0053] In this embodiment, the video frame to be identified is input into the trained event detection model for processing to determine the position of the image area where the specified event is located in the video frame to be identified, and then based on the position of the image area where the specified event is located in the video frame to be identified, the image area where the specified event is located in the video frame to be identified is obtained.

[0054] It is understandable that the event detection model can be pre-trained by the terminal device, or it can be pre-trained by other devices and then the file corresponding to the event detection model can be transplanted to the terminal device. In other words, the execution subject of training the event detection model and the execution subject of using the event detection model for event detection can be the same or different. For example, when other devices are used to train the initial event detection model, after the other devices finish training the initial event detection model, the model parameters of the initial event detection model are fixed, the file corresponding to the event detection model is obtained, and then the file is transplanted to the terminal device.

[0055] In one embodiment, sample video frames with labeled sample events are obtained; and the sample video frames with labeled sample events are used as training sets to train an event detection model.

[0056] In one embodiment, a target video is acquired, and the target video is sampled at a specified sampling rate to obtain an initial video frame. The initial video frame is input into a trained event monitoring model for processing to obtain an event detection result, which is used to describe whether the initial video frame includes an image region where a specified event is located; if the first detection result indicates that the initial video frame includes an image region where a specified event is located, the initial video frame is used as a video frame to be identified.

[0057] S13: Input the image region into the trained event classification model for processing to obtain a classification result of the specified event.

[0058] As an example of the present application, the classification result of the specified event is used to describe the classification of the specified event.

[0059] In this embodiment, in order to reduce the amount of calculation and quickly classify the video frame to be identified, after detecting the image area where the specified event is located in the video frame to be identified, the image area where the specified event is located is input into the trained event classification model for processing to obtain the classification result of the specified event.

[0060] It can be understood that in order to classify the image area where a specified event is located, when it is detected that the video frame to be identified contains the image area where the specified event is located, the image area of ​​the specified event is segmented from the video frame to be identified, and then the segmented image area is input into the trained event classification model for processing, so as to obtain the classification result of the specified event.

[0061] It is understandable that the event classification model can be pre-trained by the terminal device, or it can be pre-trained by other devices and then the file corresponding to the event classification model can be transplanted to the terminal device. In other words, the execution subject of training the event classification model and the execution subject of using the event classification model for event classification can be the same or different. For example, when other devices are used to train the initial event classification model, after the other devices finish training the initial event classification model, the model parameters of the initial event classification model are fixed, and the file corresponding to the event classification model is obtained, and then the file is transplanted to the terminal device.

[0062] In one embodiment, a sample image region with labeled sample classification is obtained, and a sample image region with labeled sample events is used as a training set to train an event classification model.

[0063] S14: According to the object detection result of the video frame to be identified, the classification result of the designated event is corrected.

[0064] As an example of the present application, the object detection result is used to describe the objects present in the video frame to be identified. Figure 2For example, object detection is performed on the video frame to be identified to determine pedestrians, vehicles, or trailers corresponding to pedestrians in the video frame to be identified.

[0065] In this embodiment, when the image area is input into the trained event classification model for processing, the classification result of the specified event may contain errors due to the inaccurate classification of the trained event classification model. Therefore, it is necessary to detect the video frame to be identified to obtain the object detection result of the video frame to be identified, so as to determine whether the object contained in the image area corresponding to the specified event is classified incorrectly based on the object situation described in the object detection result, and then correct the classification result of the specified event to obtain the correct classification result of the image area of ​​the specified event.

[0066] For example, Figure 2 As shown, Figure 2 The image area corresponding to the black frame is the image area corresponding to the specified event. The image area is input into the trained event classification model for processing, and because the vehicle-mounted materials in the image area corresponding to the black frame are considered to be randomly piled garbage, the image area corresponding to the black frame is determined to be an urban management event, that is, there is a misidentification. Therefore, in order to correct this situation, object detection is performed on the video frame to be identified, and the object detection situation in the image area corresponding to the black frame is obtained, that is, it is determined that the image area corresponding to the black frame contains people and vehicle-mounted materials, rather than urban management events.

[0067] In the event detection method provided in the embodiment of the present application, a video frame to be identified is obtained, and it is detected whether the video frame to be identified contains an image area where a specified event is located. If the image area where the specified event is located is detected in the video frame to be identified, the image area is input into a trained event classification model for processing to obtain a classification result of the specified event. Furthermore, in order to avoid misclassification of the specified event, the object detection result of the video frame to be identified is used to understand the object situation in the image area where the specified event is located, so as to facilitate the correction of the classification result of the specified event according to the object situation of the image area where the specified event is located, and obtain the correct event classification result corresponding to the image area where the specified event is located.

[0068] In one embodiment of the present application, a specific implementation method of inputting an image region into a trained event classification model for processing to obtain a classification result of a specified event may include:

[0069] If there are multiple image areas where the specified event is located in the video frame to be identified, the position difference corresponding to each image area is determined respectively. The position difference refers to the difference between the position of the image area in the video frame to be identified and the position of the image area corresponding to the image area in the previous video frame. The previous video frame refers to another video frame in the target video that contains the image area where the specified event is located.

[0070] An image region among the multiple image regions whose corresponding position difference value is smaller than a first set threshold is determined as a target image region.

[0071] The target image area is input into the event classification model for processing to obtain the classification result of the specified event.

[0072] In this embodiment, since the video frame to be identified can be a target area, there are multiple specified events in the target area, so that the video frame to be identified describing the target area includes image areas corresponding to multiple specified events. Moreover, since each video frame to be identified describes the situation of the target area at different time points in the process of continuously acquiring the video frame to be identified, the specified event in the target area may change. Therefore, in order to determine the change of the specified event in the target area, the position difference corresponding to each image area is determined respectively, that is, the position difference between the position information of the image area of ​​the specified event in the current video frame to be identified and the position information of the image area of ​​the specified event in the previous video frame is calculated. Further, the position difference corresponding to each image area is compared with the first set threshold value, and when the position difference is less than the first set threshold value, it means that the change of the specified event in the target area is not obvious, and it may be an event that needs to be monitored. Therefore, the image area with a position difference less than the first set threshold value is determined as the target image area, and the target image area is input into the event classification model for processing to obtain the classification result of the specified event.

[0073] It can be understood that when the position difference is greater than or equal to the first set threshold, it indicates that the change in the specified event in the target area is obvious and may not be an event that needs to be monitored. Therefore, the image area where the position difference is greater than or equal to the first set threshold is not the target image area.

[0074] For example, the first video frame to be identified includes image areas corresponding to the designated event A, the designated event B, and the designated event C, respectively. At the same time, the previous video frame corresponding to the first video frame to be identified also includes image areas corresponding to the designated event A, the designated event B, and the designated event C, respectively. Therefore, in order to determine the changes of the designated event A, the designated event B, and the designated event C in the first video frame to be identified, the position differences corresponding to the image areas corresponding to the designated event A, the designated event B, and the designated event C in the first video frame to be identified are determined respectively, that is, the position of the image area of ​​the designated event A included in the first video frame to be identified is subtracted from the position of the image area corresponding to the designated event A in the previous video frame to obtain the position difference corresponding to the image area of ​​the designated event A, and so on, the position difference corresponding to the image area of ​​the designated event B and the position difference corresponding to the image area of ​​the designated event C are obtained respectively.

[0075] It can be understood that when determining the position difference corresponding to each image area, the coordinates of the position of the image area in the video frame to be identified are subtracted from the coordinates of the position of the image area corresponding to the image area in the previous video frame to obtain the position difference corresponding to the image area.

[0076] In one embodiment, in order to calculate the position difference corresponding to each image area, the video frame to be identified and the previous video frame corresponding to the video frame to be identified are placed in the same coordinate system, so that the position difference corresponding to the image area is calculated according to the coordinates of the image area in the video frame to be identified and the coordinates of the image area in the corresponding previous video frame, so that the change of the specified event in the target area during the time period from the time point of obtaining the previous video frame to the time point of obtaining the video frame to be identified is determined by the position difference, and then it is determined whether the image area of ​​the specified event needs to be processed. Among them, since when the position difference is greater than or equal to the first set threshold, it means that the change of the specified event in the target area is obvious and may not be an event that needs to be monitored, so there is no need to classify the image area where the position difference is greater than or equal to the first set threshold, which can effectively improve the classification efficiency of the image area of ​​the specified event contained in the video frame to be identified.

[0077] In one embodiment, after obtaining a video frame to be identified and detecting an image region where a specified event contained in the video frame to be identified is located, the method further includes:

[0078] If the video frame to be identified is not the first video frame, the time difference between the acquisition time of the current video frame to be identified and the acquisition time of the previous video frame is determined.

[0079] Determine the position difference between the position information of the image area of ​​the specified event in the current video frame to be identified and the position information of the image area of ​​the specified event in the previous video frame.

[0080] A first set threshold is obtained according to the time difference.

[0081] An image region whose position difference is less than a first set threshold is determined as a target image region.

[0082] The target image area is input into the event classification model for processing to obtain the classification result of the specified event.

[0083] In this embodiment, since the specified event in the target area may change during the time period from the acquisition time of the previous video frame to the acquisition time of the current video frame to be identified, such as the position of the specified event in the target area moves, different first set thresholds are pre-set according to different time differences, so that the first set threshold can be used as a reference for judging the change in the position of the specified event.

[0084] In one embodiment of the present application, the object detection result of the video frame to be identified is obtained by:

[0085] The video frame to be identified is input into the trained object detection model for processing to obtain the object detection result. The object detection result is used to indicate whether there is a target object in the image area.

[0086] As an example of the present application, the target object is an object that exists in the target area and is recorded in the video frame to be identified by photographing the target area through a camera. Figure 2 Pedestrians or vehicles shown in.

[0087] In this embodiment, in order to reduce the amount of calculation and efficiently determine the object contained in the video frame to be identified, the video frame to be identified is input into a trained object detection model for processing to obtain an object detection result indicating whether there is a target object in the image area.

[0088] It is understandable that the object detection model can be pre-trained by the terminal device, or it can be pre-trained by other devices and then the file corresponding to the object detection model can be transplanted to the terminal device. In other words, the execution subject of training the object detection model and the execution subject of using the object detection model for object detection can be the same or different. For example, when other devices are used to train the initial object detection model, after the other devices finish training the initial object detection model, the model parameters of the initial object detection model are fixed, the file corresponding to the object detection model is obtained, and then the file is transplanted to the terminal device.

[0089] In one embodiment, sample video frames of labeled sample events and objects in the image area where the labeled sample events are located are obtained; the labeled sample events and sample video frames of objects in the image area where the labeled sample events are located are used as training sets to train an object detection model.

[0090] Combination Figure 3 In one embodiment of the present application, the specific implementation method of correcting the classification result of the specified event according to the object detection result of the video frame to be identified includes:

[0091] S21: If the object detection result indicates that there is a target object in the image region, respectively calculating a first area of ​​the image region and a second area occupied by the target object in the image region.

[0092] S22: Calculate an overlap parameter according to the first area and the second area.

[0093] S23: Correcting the classification result of the designated event according to the overlap parameter.

[0094] In this embodiment, the overlap parameter refers to the ratio between the overlapped portion of the second area occupied by the target object in the image region and the first area of ​​the image region and the first area.

[0095] In this embodiment, since the image area where the specified event is located may be classified incorrectly when the image area is input into the trained event classification model for processing, in order to make the classification of the image area where the specified event is located more accurate, object detection is performed on the video frame to be identified, and when the object detection result indicates that there is a target object in the image area, it means that the image area may not be the area where the specified event is located, so it is necessary to calculate the first area of ​​the image area and the second area occupied by the target object in the image area respectively, so as to calculate the overlap parameter, and then determine whether the target object is the main component of the image area through the overlap parameter, so as to infer the relationship between the target object and other parts in the image area, and then correct the classification result of the specified event.

[0096] For example, Figure 2 As shown, Figure 2 The black frame is the image area where the specified event is located. The first area corresponding to the black frame is calculated, and then the second area occupied by the person in the black frame is calculated. According to the first area corresponding to the black frame and the second area occupied by the person in the black frame, the overlap parameter is calculated. For example, if the first area is 1 square meter and the second area is 0.5 square meters, the overlap parameter calculated is 50%, which means Figure 2The person in the image is also the main component of the image area corresponding to the black frame. Therefore, it is wrong to classify the image area as an urban management event. Then, based on this inference, the classification result of the urban management event is corrected to a normal event.

[0097] In one embodiment of the present application, a specific implementation method of correcting the classification result of a specified event according to the overlap parameter includes:

[0098] If the overlap parameter is greater than or equal to the second set threshold, it is determined that the classification result of the specified event is correct.

[0099] If the overlap parameter is less than the second set threshold, it is determined that the classification result of the designated event is wrong, and the classification result of the designated event is corrected.

[0100] In this embodiment, the second set threshold can be preset according to actual needs.

[0101] It is understandable that, since the overlap parameter can be used to understand whether the target object is the main component of the image area where the specified event is located, in order to determine whether the target object is the main component of the image area when the target object exists in the image area, the overlap parameter is compared with the second set threshold. Among them, if the overlap parameter is greater than or equal to the second set threshold, it means that the target object is not the main component of the image area, and the classification result of the specified event is correct. On the contrary, if the overlap parameter is less than the second set threshold, it is determined that the classification result of the specified event is wrong, and the classification result of the specified event is corrected.

[0102] In one embodiment, after the classification result of the designated event is corrected, the corrected classification result of the designated event is fed back.

[0103] In this embodiment, when the revised classification result of the designated event is fed back, the feedback may be provided through the terminal device or through other electronic devices connected to the terminal device. For example, after the classification result of the designated event is revised, the terminal device feeds back the revised classification result of the designated event to the server.

[0104] For example, Figure 2 As shown, Figure 2 After processing the image area corresponding to the black frame in the figure, the classification result of the specified event is obtained as an urban management event. Figure 2 After object detection, it is determined that the overlap parameter corresponding to the black frame is less than the second set threshold, which means Figure 2The classification result of the corresponding designated event is wrong, but a normal event, so after the classification result of the designated event is corrected, the terminal device feeds back the corrected classification result to the server of the smart urban management system so that the urban management personnel can view the message. On the contrary, if it is determined that the overlap parameter corresponding to the black box is greater than or equal to the second set threshold, the classification result of the designated event is a city management event, and this classification result is correct, and it needs to be fed back to the server so that the urban management personnel can handle the designated event.

[0105] Among them, when the overlap parameter is less than the second set threshold, it is determined that the classification result of the specified event is wrong, that is, the event corresponding to the image area is a normal event, and no feedback may be given, that is, the classification result of the specified event is corrected, and no information push may be performed. On the contrary, if the overlap parameter is greater than or equal to the second set threshold, it is determined that the classification result of the specified event is correct. In order for the relevant management personnel to quickly handle the specified event, it is necessary to push information so that the relevant management personnel can view the information and handle the specified event.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0107] Corresponding to the event detection method of the above embodiment, Figure 5 A structural block diagram of an event detection device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0108] Reference Figure 5 , the device 100 comprises:

[0109] An acquisition module 101 is used to acquire a video frame to be identified;

[0110] A detection module 102 is used to detect an image region where a designated event is located in a video frame to be identified;

[0111] The classification module 103 is used to input the image area into the trained event classification model for processing to obtain the classification result of the specified event;

[0112] The correction module 104 is used to correct the classification result of the specified event according to the object detection result of the video frame to be identified.

[0113] In one embodiment, the detection module 102 is further used to input the video frame to be identified into a trained event detection model for processing to obtain an image region where a specified event is located in the video frame to be identified.

[0114] In one embodiment, the classification module 103 is also used to determine the position difference corresponding to each image area if there are multiple image areas where the specified event is located in the video frame to be identified, where the position difference refers to the difference between the position of the image area in the video frame to be identified and the position corresponding to the image area in the previous video frame, where the previous video frame refers to another video frame in the target video that contains the image area where the specified event is located; determine the image area among the multiple image areas whose corresponding position difference is less than a set threshold as the target image area; and input the target image area into the event classification model for processing to obtain the classification result of the specified event.

[0115] In one embodiment, the apparatus 100 further includes an object detection module.

[0116] The object detection module is used to input the video frame to be identified into the trained object detection model for processing to obtain the object detection result. The object detection result is used to indicate whether there is a target object in the image area.

[0117] In one embodiment, the correction module 104 is further used to calculate a first area of ​​the image area and a second area occupied by the target object in the image area if the object detection result indicates that there is a target object in the image area; calculate an overlap parameter based on the first area and the second area; and correct the classification result of the specified event based on the overlap parameter.

[0118] In one embodiment, the correction module 104 is also used to determine that the classification result of the specified event is correct if the overlap parameter is greater than or equal to the set threshold; if the overlap parameter is less than the set threshold, determine that the classification result of the specified event is wrong, and correct the classification result of the specified event.

[0119] In one embodiment, the acquisition module 101 is further used to acquire a target video; sample the target video at a specified sampling rate to obtain a video frame to be identified, wherein the specified sampling rate is determined according to a frame rate of the target video.

[0120] This embodiment provides an event detection device for implementing any event detection method in the method embodiments, wherein the functions of each module may refer to the corresponding description in the method embodiments, and the implementation principles and technical effects are similar, which will not be repeated here.

[0121] Figure 5 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 5 As shown, the terminal device 5 of this embodiment includes: at least one processor 50 ( Figure 5Only one processor is shown in the figure), a memory 51, and a computer program 52 stored in the memory 51 and executable on at least one processor 50. When the processor 50 executes the computer program 52, the steps in any of the above-mentioned event detection method embodiments are implemented.

[0122] The terminal device 5 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the terminal device 5 and does not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0123] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0124] In some embodiments, the memory 51 may be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. In other embodiments, the memory 51 may also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the terminal device 5. The memory 51 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0125] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0126] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0127] An embodiment of the present application also provides a terminal device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on at least one processor, and when the processor executes the computer program, the steps in any of the above-mentioned method embodiments are implemented.

[0128] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0129] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can be executed by the processor. The steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An event detection method, characterized in that: include: Get the video frame to be recognized; Detecting an image region where a designated event contained in the to-be-identified video frame is located; Inputting the image region into a trained event classification model for processing to obtain a classification result of the designated event; Correcting the classification result of the designated event according to the object detection result of the video frame to be identified; The step of inputting the image region into a trained event classification model for processing to obtain a classification result of the designated event includes: If there are multiple image regions where the designated event is located in the video frame to be identified, then respectively determine the position difference value corresponding to each of the image regions, wherein the position difference value refers to the difference between the position of the image region in the video frame to be identified and the position corresponding to the image region in the previous video frame, wherein the previous video frame refers to another video frame in the target video that contains the image region where the designated event is located; Determine an image area in the plurality of image areas where the corresponding position difference is less than a first set threshold as a target image area; The target image area is input into the event classification model for processing to obtain a classification result of the designated event.

2. The event detection method according to claim 1, characterized in that: Detecting the image region where the designated event contained in the to-be-identified video frame is located includes: The video frame to be identified is input into a trained event detection model for processing to obtain an image region where the designated event is located in the video frame to be identified.

3. The event detection method according to claim 1, characterized in that: The object detection result of the video frame to be identified is obtained by: The video frame to be identified is input into a trained object detection model for processing to obtain the object detection result, and the object detection result is used to indicate whether there is a target object in the image area.

4. The event detection method according to claim 3, characterized in that: The step of correcting the classification result of the designated event according to the object detection result of the video frame to be identified includes: If the object detection result indicates that the target object exists in the image area, respectively calculating a first area of ​​the image area and a second area occupied by the target object in the image area; Calculating an overlap parameter according to the first area and the second area; The classification result of the designated event is modified according to the overlap parameter.

5. The event detection method according to claim 4, characterized in that: The classification result of the designated event is modified according to the overlap parameter, including: If the overlap parameter is greater than or equal to a second set threshold, it is determined that the classification result of the designated event is correct; If the overlap parameter is less than the second set threshold, it is determined that the classification result of the designated event is wrong, and the classification result of the designated event is corrected.

6. The event detection method according to any one of claims 1 to 5, characterized in that: The step of obtaining a video frame to be identified includes: Get the target video; The target video is sampled according to a specified sampling rate to obtain the video frame to be identified, wherein the specified sampling rate is determined according to the frame rate of the target video.

7. An event detection device, characterized in that: include: An acquisition module, used for acquiring a video frame to be identified; A detection module, used for detecting an image region where a designated event contained in the video frame to be identified is located; A classification module, used for inputting the image area into a trained event classification model for processing to obtain a classification result of the specified event; A correction module, used to correct the classification result of the specified event according to the object detection result of the video frame to be identified; Wherein, the classification module is specifically used for: If there are multiple image regions where the designated event is located in the video frame to be identified, then respectively determine the position difference value corresponding to each of the image regions, wherein the position difference value refers to the difference between the position of the image region in the video frame to be identified and the position corresponding to the image region in the previous video frame, wherein the previous video frame refers to another video frame in the target video that contains the image region where the designated event is located; Determine an image area in the plurality of image areas where the corresponding position difference is less than a first set threshold as a target image area; The target image area is input into the event classification model for processing to obtain a classification result of the designated event.

8. A terminal device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the event detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the event detection method according to any one of claims 1 to 6 is implemented.

Images