Data processing method, device, storage medium and computer equipment

Abstract

The present application discloses a data processing method, device, storage medium and computer equipment. The method includes: obtaining target segmented data of the video data to be processed, the target segmented data includes key frames; identifying the type of the key frames to obtain a first recognition result; determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; correcting the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data. The present application solves the technical problem that the prior art has inaccurate results for video type recognition.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data processing method, apparatus, storage medium and computer equipment. Background Art

[0002] In the field of video understanding, automatic recognition of people, objects, scenes, etc. in videos is a key technology for video structuring. Scene is a very important dimension in video information, which can be used to generate cover images and insert commercial advertisements in subsequent video production. Due to the huge amount of video inventory and the long duration of drama videos, the workload of manual screening is very heavy. Therefore, a common method is to use deep learning to train an image-based scene recognition type detector to perform scene recognition on the video frame by frame.

[0003] However, the discrimination method based on single-frame images has certain limitations. First, the efficiency of single-frame video processing is low. More importantly, due to the limited viewing angle of the image, there are often ambiguous judgments when performing scene recognition, and it is not easy to identify which scene the current image belongs to. For example, the two scenes of a hotel room and a family bedroom are very similar, and in many cases it is difficult to determine whether it belongs to a bedroom or a hotel room. For example, if there is only one person sitting on the sofa in the picture, it is difficult to determine whether it belongs to a living room or a studio based on the limited information.

[0004] With respect to the problem that the above-mentioned prior art has inaccurate results in identifying the type of videos, no effective solution has been proposed so far. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, apparatus, storage medium and computer device to at least solve the technical problem of inaccurate video type recognition results in the prior art.

[0006] According to one aspect of an embodiment of the present application, a data processing method is provided, comprising: obtaining target segmented data of video data to be processed, the target segmented data comprising key frames; identifying the type of the key frames to obtain a first recognition result; determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; and correcting the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0007] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining target segmented data of multimedia data to be processed, wherein the target segmented data is segmented data obtained after segmentation processing of the multimedia data; identifying the type of the target segmented data to obtain a first recognition result; and correcting the first recognition result based on the type recognition result of other segmented data in the multimedia data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0008] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining target segmented data of audio data to be processed, wherein the target segmented data is segmented data obtained after segmentation processing of the audio data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result according to the types of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0009] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining target segmented data of text data to be processed, wherein the target segmented data is segmented data obtained after segmenting the text data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result based on the types of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0010] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining target segmented data of an image sequence to be processed, wherein the target segmented data is segmented data obtained after segmentation processing of the image sequence data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result according to the types of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0011] According to another aspect of an embodiment of the present application, a data processing device is also provided, including: an acquisition module, used to acquire target segmented data of video data to be processed, the target segmented data including key frames; an identification module, used to identify the type of the key frames to obtain a first identification result; a determination module, used to determine other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; a correction module, used to correct the first identification result according to the other segmented data to obtain a second identification result, wherein the second identification result is used to indicate the type of the target segmented data.

[0012] According to another aspect of an embodiment of the present application, a storage medium is further provided, wherein the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above-mentioned data processing method.

[0013] According to another aspect of an embodiment of the present application, a computing device is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: obtaining target segmented data of video data to be processed, the target segmented data including key frames; identifying the type of the key frames to obtain a first recognition result; determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; correcting the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0014] In an embodiment of the present application, video data to be processed is obtained, the video data to be processed is divided into a plurality of segmented data, and a key frame of each segmented data is determined, then target segmented data in the video data to be processed is selected, and the type of the key frame corresponding to the target segmented data is identified to obtain a first recognition result, and then based on the types of other segmented data in the video data to be processed, the first recognition result identified based on the key frame of the target segmented data is corrected to obtain a second recognition result of the target segmented data, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of the plurality of segmented data, thereby achieving the technical effect of accurately identifying the type in the video data to be processed, and further solving the technical problem of inaccurate type recognition results of the video in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a computer terminal for implementing a data processing method is shown;

[0017] Figure 2 A data processing method according to an embodiment of the present application Figure 1 ;

[0018] Figure 3a is a schematic diagram of a scene (ie, type) recognition system according to an embodiment of the present application;

[0019] Figure 3b is a schematic diagram of system execution steps for scene (ie, type) identification according to an embodiment of the present application;

[0020] Figure 4 A data processing method according to an embodiment of the present application Figure 2 ;

[0021] Figure 5 is a flowchart of a data processing method according to an embodiment of the present application;

[0022] Figure 6 A data processing method according to an embodiment of the present application Figure 4 ;

[0023] Figure 7 A data processing method according to an embodiment of the present application Figure 5 ;

[0024] Figure 8 A data processing device according to an embodiment of the present application is shown in FIG. Figure 1 ;

[0025] Fig. 9 A data processing device according to an embodiment of the present application is shown in FIG. Figure 2 ;

[0026] Fig.10 is a third schematic diagram of a data processing device according to an embodiment of the present application;

[0027] Fig.11 A data processing device according to an embodiment of the present application is shown in FIG. Figure 4 ;

[0028] Fig.12 A data processing device according to an embodiment of the present application is shown in FIG. Figure 5 ;

[0029] Fig.13 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0033] Scene recognition: Determine what kind of scene or place appears in the picture. Common scenes include living room, bedroom, kitchen, classroom, swimming pool, etc.

[0034] Shot size: The size and range of the subject and image in the frame structure of the TV screen. It is divided into close shot, medium shot, long shot, etc., and is usually judged based on the proportion of the face in the image.

[0035] Convolutional Neural Networks: A deep learning approach for learning image features.

[0036] Example 1

[0037] According to an embodiment of the present application, a data processing method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the data processing method of the above-mentioned application is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, this application provides Figure 2 A data processing method is shown. Figure 2 A data processing method according to an embodiment of the present application Figure 1 ,like Figure 2 As shown, the method comprises the following steps:

[0044] Step S202, obtaining target segmented data of the video data to be processed, wherein the target segmented data includes key frames;

[0045] Step S204, identifying the type of the key frame to obtain a first identification result;

[0046] Step S206, determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data;

[0047] Step S208: modify the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0048] In an embodiment of the present application, video data to be processed is obtained, the video data to be processed is divided into at least two segmented data with a time sequence relationship, and the key frames of each segmented data are determined, and then the target segmented data in the video data to be processed is selected, and the type of the key frame corresponding to the target segmented data is identified to obtain a first recognition result, and then the type recognition results of N segmented data located in the target segmented data in the video data to be processed and the correlation relationship between the target segmented data and the type recognition results of the N segmented data are used to correct the first recognition result identified based on the key frame of the target segmented data to obtain a second recognition result of the target segmented data, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the video data to be processed, and further solving the technical problem of inaccurate type recognition results of the video in the prior art.

[0049] In the above step S202, the video data to be processed may be a video file such as a movie or a TV series, the target segmented data may be a segment of a video file such as a TV series or a TV series, and each target segmented data includes at least two segmented data having a time sequence relationship.

[0050] For example, if the content of the target segmented data shows that a person enters a bedroom from a living room, the type in the video may change from a living room to a bedroom, and the segmented data under the living room and the segmented data under the bedroom are segmented data having a time series relationship.

[0051] It should be noted that videos are composed of still images. When the image changes more than 24 frames per second, such continuous images are called videos, and each still image is called a frame.

[0052] Therefore, each segmented data includes multiple frames, and a frame of picture that best represents the type of the segmented data is selected from each segmented data as the key frame of the segmented data.

[0053] Optionally, an image frame with the largest background image and the most type features may be selected as a key frame.

[0054] As an optional embodiment, the target segmented data of the video data to be processed may be obtained in the following manner: using a shot boundary detection technology to perform shot segmentation processing on the target segmented data to obtain a plurality of shot data; and determining the target segmented data from the plurality of shot data.

[0055] In the above-mentioned embodiment of the present application, a plurality of shot data can be obtained through segmentation processing, and then target segment data can be selected from the plurality of shot data, thereby completing the selection of the target segment data.

[0056] In some embodiments of the present application, the type of key frame includes: the scene of the key frame; in order to save computing resources, before identifying the type of the key frame, the key frame can be judged, specifically: determining the shot to which the key frame belongs; judging whether the shot is the target shot type; when the shot is the target shot type, determining to allow identification of the type of the key frame; when the shot is not the target shot type, determining to refuse to identify the type of the key frame.

[0057] Optionally, the scenes belonging to the target scene type include: medium shot or long shot; the scenes not belonging to the target scene type include: close shot or close-up. Before identifying the type of the key frame, the scene to which the key frame belongs can be identified, that is, whether each frame image in the segmented data belongs to a close-up or close-up, or a medium shot or long shot, and then the type of the key frame belonging to the medium shot or long shot can be identified.

[0058] As an optional embodiment, determining the scene size to which the key frame belongs includes: determining the proportion of the size of the target object in the picture corresponding to the key frame; determining the value range to which the proportion belongs; determining the scene size corresponding to the value range, and using the scene size corresponding to the value range as the scene size to which the key frame belongs.

[0059] In the above embodiments of the present application, the target object can be the main content of the video, such as people and animals. The larger the area occupied by the target object in the picture, the closer the target object is to the camera lens. The scene scale of the key frame can be determined by identifying the proportion of the size of the target object in the key frame picture.

[0060] For example, you can set the ratio range corresponding to the mid-shot or long shot, and the ratio range corresponding to the close-up or close-up, so that you can judge the scene type of the key frame based on the ratio of the size of the target object in the key frame to the key frame picture.

[0061] As an optional embodiment, identifying the type of a key frame includes: inputting an image corresponding to the key frame into a neural network model for analysis to obtain a first recognition result, wherein the neural network model is trained using multiple sets of type annotation data.

[0062] The above-mentioned embodiments of the present application can be based on image recognition technology and pre-train a neural network model for type recognition using multiple groups of type annotation data. In the process of identifying the type of key frames, the key frames can be input into the neural network model, and the neural network model can complete the type recognition of the key frames.

[0063] In some embodiments of the present application, the above-mentioned other segmented data include but are not limited to data located before the target segmented data in the video data to be processed, and may also be data located after the target segmented data in the video data to be processed, and may also include segmented data located before and after the target segmented data in the video data to be processed. Wherein, for the case of data located before the target segmented data in the video data to be processed, determining the type of the other segmented data in the video data to be processed includes: determining the other segmented data in the video data to be processed, including: determining N segmented data located before the target segmented data in the video data to be processed, wherein N is a positive integer; at this time, when the first recognition result is corrected according to the other segmented data, the type of the N segmented data can be determined; and according to the association relationship between the types of the N segmented data, the first recognition result is corrected.

[0064] As an optional embodiment, the first recognition result is corrected according to the association relationship between the types of N segmented data, including: determining the reference type to which the target segmented data belongs according to the association relationship between the types of N segmented data; and correcting the first recognition result according to the reference type to which the target segmented data belongs.

[0065] If a bed appears in the picture of a key frame, the type of the key frame can be limited to indoor based on type recognition of the key frame. For example, the type of the key frame can be determined to be a home bedroom and a hotel room. In this case, the type of the current key frame can be judged based on the type recognition results of the first N segmented data of the segmented data to which the current key frame belongs. For example, if the type recognition results of the first N segmented data of the segmented data to which the current key frame belongs are a living room, then it can be determined that the recognition result of the current key frame is a home bedroom.

[0066] As an optional embodiment, determining the reference type to which the target segmented data belongs according to the association relationship between the types of the N segmented data includes: determining the types of the N segmented data to obtain multiple types; determining a common type to which the multiple types belong; and determining the reference type to which the target segmented data belongs according to the subordinate relationship between the common type and the multiple types. The above subordinate relationship includes but is not limited to: multiple types are subordinate to a common type, for example, the types of kitchen and living room are subordinate to the family type.

[0067] It should be noted that the public type may include multiple types. For example, the public type may be a family room, and the multiple types under the public type include the bedroom, living room, kitchen, dining room, bathroom, entrance hall, etc. in the family room.

[0068] This application proposes a system for effectively identifying scenes (i.e., types) in videos, which combines scene recognition with contextual information and expert knowledge on video timing, and can effectively handle ambiguous judgments. In addition, the system only recognizes the key frames of each shot after the video is divided into sections, which can further improve the efficiency of recognition.

[0069] Figure 3a is a schematic diagram of a scene (ie, type) recognition system according to an embodiment of the present application, such as Figure 3a As shown, the scene recognition system may include: a video segmentation module 32, a scene recognition module 34, a scene identifier 36, and an association identifier 38, wherein the video segmentation module 32 is used to perform scene recognition on the video data to be processed, and obtain multiple key frames with a temporal relationship in the video to be processed; the scene recognition module 34 is used to identify the scene of the key frame and select the key frame belonging to the middle view or the long view; the scene identifier 36 is used to perform scene recognition on the key frame of the middle view or the long view; the association identifier 38 is used to correct the scene recognition result of the current key frame based on the scene recognition results of multiple key frames with a temporal association relationship.

[0070] Figure 3b is a schematic diagram of a system execution step for scene (ie, type) recognition according to an embodiment of the present application, such as Figure 3b As shown, the specific process of the program is as follows:

[0071] Step 1) performs segmentation processing on the input video using shot boundary detection technology to obtain multiple shots, which are sequentially sent to step 2).

[0072] 2) Extract key frames for the current shot.

[0073] 3) Perform scene recognition on the extracted key frames.

[0074] 4) If the scene is identified as a close-up or a close-up, the key frame will not be identified as a scene, and the process will go back to step 2).

[0075] 5) If the scene is identified as medium or long shot, the key frame is sent to the scene recognizer for identification.

[0076] 6) Based on the recognition results of N shots and scene expert knowledge, a secondary judgment is made on the recognition result of the shot.

[0077] 7) Output the final recognition result and store it, jump back to step 2) to judge the key frame of the next shot.

[0078] It should be noted that in the above step 3), "scene size" refers to the size and range of the subject and the image in the TV screen frame structure, which is divided into close-up, mid-shot, and long-shot, etc., and is usually judged based on the proportion of the face in the image.

[0079] Optionally, the "scene recognition" process in the above step 3) is judged based on the face detection result. In this solution, face detection uses the retina face face detection algorithm. When the height of the face in the key frame of the video is greater than 1 / 2 of the screen height, the scene of the key frame is judged to be a close-up; when the height of the face is between 1 / 4-1 / 2 of the screen height, the scene of the key frame is judged to be a close-up; when the height of the face is less than 1 / 4 of the screen height, the scene of the key frame is judged to be a medium shot or a distant shot.

[0080] It should be noted that a scene is usually a partial overall description and summary of a place or situation, and close-up and near-shot images cannot well express the overall information of the current scene. Therefore, scene recognition in this case is prone to ambiguity and is not suitable for recognition, so it is not processed. If the current frame is a mid-shot or a long shot, it is sent to the scene recognition type for scene recognition.

[0081] It should be noted that the scene recognition type detector in the above step 5) is obtained by training based on the convolutional neural network using the precisely labeled scene annotation data. The convolutional neural network adopts the ResNet50 structure, and the input image size is 224*224. Through the ResNet50 network, a 2048-dimensional feature vector is extracted, and then the vector is connected to the fc layer and the softmax layer to output the final type label.

[0082] It should be noted that due to the limitations of the key frame perspective of static images, the judgment results for a single key frame are often ambiguous. For example, it is easy to confuse a home bedroom with a hotel room. This application uses the timing information of the video and considers the continuity of the plot of the series. The recognition results of this shot are re-judged based on the recognition results of N shots and expert knowledge.

[0083] It should be noted that expert knowledge is the temporal correlation between scenes generated based on knowledge graphs and common sense. For example, the living room, bedroom, and kitchen scenes are all home scenes, and the physical distance is very close, so the corresponding correlation is very high; in contrast, the correlation between the living room and hotel room scenes is weaker. When identifying scenes, for two scenes that are easily confused, a secondary judgment can be made based on the temporal correlation of N scenes. For example, it is difficult to determine whether the current shot is a hotel room or a living room or bedroom. If a home scene such as a living room appears in N shots, we believe that the shot is more likely to be a bedroom. In this solution, N is 6. The recognition results of N shots are maintained by a queue. When the recognition results of a new shot are stored, the recognition results of the earliest inserted shot are deleted.

[0084] The technical solution provided in this application first adopts scene recognition to filter out close-ups and close-ups, which are images that are likely to cause ambiguity in scene recognition; then the timing information of the video and the correlation between the scenes are used to make a secondary judgment on the scene recognition results, thereby improving the accuracy of scene recognition.

[0085] The technical solution provided in the present application processes the video using storyboards and key frames and then identifies the key frames, which is more efficient than frame-by-frame identification processing.

[0086] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0087] Under the above operating environment, this application provides Figure 4 A data processing method is shown. Figure 4 A data processing method according to an embodiment of the present application Figure 2 ,like Figure 4 As shown, the method comprises the following steps:

[0088] Step S402, obtaining target segmented data of the multimedia data to be processed, wherein the target segmented data is segmented data obtained after segmenting the multimedia data;

[0089] Step S404, identifying the type of the target segmented data to obtain a first identification result;

[0090] Step S406: modify the first recognition result according to the type recognition results of other segmented data in the multimedia data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0091] It should be noted that the other segmented data in the above step S406 are adjacent in time sequence.

[0092] In an embodiment of the present application, multimedia data to be processed is obtained, and the multimedia data to be processed is divided into at least two segmented data with a time sequence relationship, and then the target segmented data in the multimedia data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the multimedia data to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0093] Under the above operating environment, this application provides Figure 5 A data processing method is shown. Figure 5 Flowchart 3 of a data processing method according to an embodiment of the present application, Figure 5 As shown, the method comprises the following steps:

[0094] Step S502, obtaining target segmented data of the audio data to be processed, wherein the target segmented data is segmented data obtained after segmenting the audio data;

[0095] Step S504, identifying the type of the target segmented data to obtain a first identification result of the target segmented data;

[0096] Step S506: modify the first recognition result according to the type of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0097] In an embodiment of the present application, audio data to be processed is obtained, and the audio data to be processed is divided into at least two segmented data with a time sequence relationship, and then the target segmented data in the audio data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the audio data to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0098] Under the above operating environment, this application provides Figure 6 A data processing method is shown. Figure 6 A data processing method according to an embodiment of the present application Figure 4,like Figure 6 As shown, the method comprises the following steps:

[0099] Step S602, obtaining target segmented data of the text data to be processed, wherein the target segmented data is segmented data obtained after segmenting the text data;

[0100] Step S604, identifying the type of the target segmented data to obtain a first identification result of the target segmented data;

[0101] Step S606: modify the first recognition result according to the type of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0102] In an embodiment of the present application, text data to be processed is obtained, the text data to be processed is divided into at least two segmented data with a time sequence relationship, then the target segmented data in the text data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the text data to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0103] Under the above operating environment, this application provides Figure 7 A data processing method is shown. Figure 7 A data processing method according to an embodiment of the present application Figure 5 ,like Figure 7 As shown, the method comprises the following steps:

[0104] Step S702, obtaining target segmented data of the image sequence to be processed, wherein the target segmented data is segmented data obtained after segmenting the image sequence data;

[0105] Step S704, identifying the type of the target segmented data to obtain a first identification result of the target segmented data;

[0106] Step S706: modify the first recognition result according to the type of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0107] In an embodiment of the present application, a sequence of images to be processed is obtained, the sequence of images to be processed is divided into at least two segmented data having a temporal relationship, then target segmented data in the sequence of images to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the sequence of images to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data having a temporal relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0109] Example 2

[0110] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 8 A data processing device according to an embodiment of the present application is shown in FIG. Figure 1 ,like Figure 8 As shown, the device includes: an acquisition module 82 , an identification module 84 , a determination module 86 and a correction module 88 .

[0111] Among them, the acquisition module 82 is used to obtain the target segmented data of the video data to be processed, and the target segmented data includes key frames; the identification module 84 is used to identify the type of the key frame to obtain a first recognition result; the determination module 86 is used to determine other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; the correction module 88 is used to correct the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0112] It should be noted that the acquisition module 82, identification module 84, determination module 86 and correction module 88 correspond to step S202, step S204, step S206 and step S208 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0113] In an embodiment of the present application, video data to be processed is obtained, the video data to be processed is divided into at least two segmented data with a time sequence relationship, and the key frames of each segmented data are determined, and then the target segmented data in the video data to be processed is selected, and the type of the key frame corresponding to the target segmented data is identified to obtain a first recognition result, and then the type recognition results of N segmented data located in the target segmented data in the video data to be processed and the correlation relationship between the target segmented data and the type recognition results of the N segmented data are used to correct the first recognition result identified based on the key frame of the target segmented data to obtain a second recognition result of the target segmented data, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the video data to be processed, and further solving the technical problem of inaccurate type recognition results of the video in the prior art.

[0114] As an optional embodiment, the type includes: the scene of the key frame; the device may also include: a first determination submodule, used to determine the scene to which the key frame belongs before identifying the type of the key frame; a judgment module, used to judge whether the scene is a target type; a second determination submodule, used to determine that the scene of the key frame is allowed to be identified when the scene is the target type; and when the scene is not the target type, determine that the scene of the key frame is rejected.

[0115] As an optional embodiment, the first determination submodule includes: a third determination submodule, used to determine the proportion of the size of the target object in the picture corresponding to the key frame; a fourth determination submodule, used to determine the value range to which the proportion belongs; a fifth determination submodule, used to determine the scene size corresponding to the value range, and use the scene size corresponding to the value range as the scene size to which the key frame belongs.

[0116] As an optional embodiment, the recognition module includes: an analysis submodule, which is used to input the image corresponding to the key frame into the neural network model for analysis to obtain a first recognition result, wherein the neural network model is trained using multiple groups of type annotation data.

[0117] As an optional embodiment, the determination module includes: a sixth determination submodule, used to determine N segmented data located before the target segmented data in the video data to be processed, where N is a positive integer; the correction module includes: a first correction submodule, used to determine the type to which the N segmented data belong; and correcting the first recognition result based on the correlation between the types of the N segmented data.

[0118] As an optional embodiment, the first correction submodule includes: a seventh determination submodule, used to determine the reference type to which the target segmented data belongs based on the association relationship between the types of N segmented data; and a second correction submodule, used to correct the first recognition result based on the reference type to which the target segmented data belongs.

[0119] As an optional embodiment, the seventh determination submodule includes: an eighth determination submodule, used to determine the types of N segmented data to obtain multiple types; a ninth determination submodule, used to determine the common type to which the multiple types belong; and a tenth determination submodule, used to determine the reference type to which the target segmented data belongs based on the subordinate relationship between the common type and the multiple types.

[0120] As an optional embodiment, the acquisition module includes: a frame processing module for performing frame processing on the video data to be processed using a shot boundary detection technology to obtain multiple shot data; and an eleventh determination module for determining target segmented data from the multiple shot data.

[0121] As an optional embodiment, the device also includes: a display module, used to display the first recognition result to the target object before determining other segmented data in the video data to be processed; a receiving module, used to receive feedback information fed back by the target object based on the first recognition result; and a triggering module, used to trigger other segmented data when the feedback information indicates that the type indicated by the first recognition result is an invalid type.

[0122] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig. 9 A data processing device according to an embodiment of the present application is shown in FIG. Figure 2 ,like Fig. 9 As shown, the device includes: a multimedia data acquisition module 92, a multimedia data identification module 94 and a multimedia data correction module 96.

[0123] Among them, the multimedia data acquisition module 92 is used to obtain target segmented data of the multimedia data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the multimedia data; the multimedia data identification module 94 is used to identify the type of the target segmented data to obtain a first identification result; the multimedia data correction module 96 is used to correct the first identification result according to the type identification results of other segmented data in the multimedia data to obtain a second identification result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0124] It should be noted that the multimedia data acquisition module 92, multimedia data identification module 94 and multimedia data correction module 96 correspond to step S402, step S404 and step S406 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0125] In an embodiment of the present application, multimedia data to be processed is obtained, and the multimedia data to be processed is divided into at least two segmented data with a time sequence relationship, and then the target segmented data in the multimedia data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the multimedia data to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0126] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.10 Flowchart 3 of a data processing method according to an embodiment of the present application, Fig.10 As shown, the device includes: an audio data acquisition module 1002 , an audio data identification module 1004 and an audio data correction module 1006 .

[0127] Among them, the audio data acquisition module 1002 is used to obtain target segmented data of the audio data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the audio data; the audio data identification module 1004 is used to identify the type of the target segmented data and obtain a first recognition result of the target segmented data; the audio data correction module 1006 is used to correct the first recognition result according to the type of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0128] It should be noted that the above-mentioned audio data acquisition module 1002, audio data identification module 1004 and audio data correction module 1006 correspond to step S502, step S504 and step S506 in Example 1, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules as part of the device can be run in the computer terminal 10 provided in Example 1.

[0129] In an embodiment of the present application, audio data to be processed is obtained, and the audio data to be processed is divided into at least two segmented data with a time sequence relationship, and then the target segmented data in the audio data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the audio data to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0130] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.11 A data processing device according to an embodiment of the present application is shown in FIG. Figure 4 ,like Fig.11 As shown, the device includes: a text data acquisition module 1102 , a text data recognition module 1104 and a text data correction module 1106 .

[0131] Among them, the text data acquisition module 1102 is used to obtain target segmented data of the text data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the text data; the text data identification module 1104 is used to identify the type of the target segmented data and obtain a first recognition result of the target segmented data; the text data correction module 1106 is used to correct the first recognition result according to the type of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0132] It should be noted that the above-mentioned text data acquisition module 1102, text data recognition module 1104 and text data correction module 1106 correspond to step S602, step S604 and step S606 in Example 1, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules as part of the device can be run in the computer terminal 10 provided in Example 1.

[0133] In an embodiment of the present application, text data to be processed is obtained, the text data to be processed is divided into at least two segmented data with a time sequence relationship, then the target segmented data in the text data to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the text data to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0134] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.12 A data processing device according to an embodiment of the present application is shown in FIG. Figure 5 ,like Fig.12 As shown, the device includes: an image sequence acquisition module 1202 , an image sequence recognition module 1204 , and an image sequence correction module 1206 .

[0135] Among them, the image sequence acquisition module 1202 is used to obtain the target segmented data of the image sequence to be processed, wherein the target segmented data is the segmented data obtained after segmenting the image sequence data; the image sequence recognition module 1204 is used to identify the type of the target segmented data and obtain a first recognition result of the target segmented data; the image sequence correction module 1206 is used to correct the first recognition result according to the type of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0136] It should be noted that the above-mentioned image sequence acquisition module 1202, image sequence recognition module 1204, and image sequence correction module 1206 correspond to step S702, step S704, and step S706 in Example 1, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0137] In an embodiment of the present application, a sequence of images to be processed is obtained, the sequence of images to be processed is divided into at least two segmented data having a temporal relationship, then target segmented data in the sequence of images to be processed is selected, and the type of the target segmented data is identified to obtain a first recognition result, and then the first recognition result is corrected based on the type recognition results of other segmented data associated with the target segmented data in the sequence of images to be processed to obtain a second recognition result, thereby achieving the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data having a temporal relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and further solving the technical problem of inaccurate type recognition results of videos in the prior art.

[0138] Example 3

[0139] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0140] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0141] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining target segmented data of the video data to be processed, the target segmented data including key frames; identifying the type of the key frames to obtain a first recognition result; determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; correcting the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0142] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining target segmented data of the multimedia data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the multimedia data; identifying the type of the target segmented data to obtain a first recognition result; and correcting the first recognition result based on the type recognition results of other segmented data in the multimedia data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0143] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining target segmented data of the audio data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the audio data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result according to the types of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0144] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining target segmented data of the text data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the text data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result according to the types of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0145] In this embodiment, the computer terminal can execute the program code of the following steps in the data processing method: obtaining target segmented data of the image sequence to be processed, wherein the target segmented data is segmented data obtained after segmenting the image sequence data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result according to the types of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0146] Optionally, Figure 6 is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 6 As shown, the computer terminal 10 may include: one or more (only one is shown in the figure) processors, a memory, and a transmission module.

[0147] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target segmented data of the video data to be processed, wherein the target segmented data includes key frames; identify the type of the key frames to obtain a first recognition result; determine other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; correct the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0149] Optionally, the type includes: a scene of a key frame, and the processor may also execute the following program code steps: before identifying the type of the key frame, determining the scene type to which the key frame belongs; judging whether the scene type is a target type; when the scene type is the target type, determining to allow identification of the scene of the key frame; and when the scene type is not the target type, determining to refuse identification of the scene of the key frame.

[0150] Optionally, the processor may also execute program code of the following steps: determining the proportion of the size of the target object in the picture corresponding to the key frame; determining the value range to which the proportion belongs; determining the scene size corresponding to the value range, and using the scene size corresponding to the value range as the scene size to which the key frame belongs.

[0151] Optionally, the processor may also execute the program code of the following steps: inputting the image corresponding to the key frame into the neural network model for analysis to obtain the first recognition result, wherein the neural network model is trained using multiple sets of type annotation data.

[0152] Optionally, the processor may also execute the program code of the following steps: determining N segmented data in the video data to be processed that are located before the target segmented data, where N is a positive integer; determining the type to which the N segmented data belong; and correcting the first recognition result based on the correlation between the types of the N segmented data.

[0153] Optionally, the processor may further execute program codes of the following steps: determining the reference type to which the target segmented data belongs according to the association relationship between the types of the N segmented data; and correcting the first recognition result according to the reference type to which the target segmented data belongs.

[0154] Optionally, the processor may also execute program code of the following steps: determining types of the N segmented data to obtain multiple types; determining a common type to which the multiple types belong; and determining a reference type to which the target segmented data belongs based on a subordinate relationship between the common type and the multiple types.

[0155] Optionally, the processor may also execute program codes of the following steps: performing segmentation processing on the video data to be processed by using a shot boundary detection technology to obtain a plurality of shot data; and determining the target segmentation data from the plurality of shot data.

[0156] Optionally, the processor may also execute program code for the following steps: displaying the first recognition result to the target object before determining other segmented data in the video data to be processed; receiving feedback information from the target object based on the first recognition result; and triggering the other segmented data when the feedback information indicates that the type indicated by the first recognition result is an invalid type.

[0157] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target segmented data of the multimedia data to be processed, wherein the target segmented data is segmented data obtained after segmenting the multimedia data; identify the type of the target segmented data to obtain a first recognition result; and correct the first recognition result based on the type recognition results of other segmented data in the multimedia data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0158] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target segmented data of the audio data to be processed, wherein the target segmented data is segmented data obtained after segmenting the audio data; identify the type of the target segmented data to obtain a first recognition result of the target segmented data; and correct the first recognition result according to the types of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0159] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target segmented data of the text data to be processed, wherein the target segmented data is the segmented data obtained after segmenting the text data; identify the type of the target segmented data to obtain a first recognition result of the target segmented data; and correct the first recognition result according to the types of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is the segmented data associated with the target segmented data.

[0160] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target segmented data of the image sequence to be processed, wherein the target segmented data is segmented data obtained after segmenting the image sequence data; identify the type of the target segmented data to obtain a first recognition result of the target segmented data; and correct the first recognition result according to the type of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0161] By adopting the embodiment of the present application, a data processing scheme is provided. By acquiring the multimedia data to be processed, the multimedia data to be processed is divided into at least two segmented data with a time sequence relationship, and the key frame of each segmented data is determined, and then the target segmented data in the multimedia data to be processed is selected, and the type of the key frame corresponding to the target segmented data is identified to obtain a first recognition result, and then the type recognition results of N segmented data located in the target segmented data in the multimedia data to be processed and the association relationship between the type recognition results of the target segmented data and the N segmented data are corrected to obtain the second recognition result of the target segmented data, so as to achieve the purpose of jointly determining the type of the target segmented data based on the type recognition results of multiple segmented data with a time sequence relationship, thereby achieving the technical effect of accurately identifying the type in the multimedia data to be processed, and then solving the technical problem of inaccurate type recognition results of the video in the prior art.

[0162] It can be understood by those skilled in the art that Fig.13 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.13 The structure of the electronic device is not limited. For example, the computer terminal 10 may also include Fig.13 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.13 Different configurations shown.

[0163] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0164] Example 4

[0165] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided in the first embodiment.

[0166] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0167] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: obtaining target segmented data of the video data to be processed, the target segmented data including key frames; identifying the type of the key frames to obtain a first recognition result; determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; correcting the first recognition result based on the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data.

[0168] Optionally, in this embodiment, the type includes: the scene of the key frame, and the storage medium is configured to store a program code for executing the following steps: before identifying the type of the key frame, determining the shot to which the key frame belongs; judging whether the shot is a target type; when the shot is the target type, determining to allow the scene of the key frame to be identified; when the shot is not the target type, determining to refuse to identify the scene of the key frame.

[0169] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: determining the proportion of the size of the target object in the picture corresponding to the key frame; determining the value range to which the proportion belongs; determining the scene size corresponding to the value range, and using the scene size corresponding to the value range as the scene size to which the key frame belongs.

[0170] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: inputting an image corresponding to a key frame into a neural network model for analysis to obtain a first recognition result, wherein the neural network model is trained using multiple sets of type annotation data.

[0171] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: determining N segmented data located before the target segmented data in the video data to be processed, where N is a positive integer; determining the types to which the N segmented data belong; and correcting the first recognition result based on the association relationship between the types of the N segmented data.

[0172] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: determining a reference type to which the target segmented data belongs based on an association relationship between types of N segmented data; and correcting the first recognition result based on the reference type to which the target segmented data belongs.

[0173] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: determining types of N segmented data to obtain multiple types; determining a common type to which the multiple types belong; and determining a reference type to which the target segmented data belongs based on a subordinate relationship between the common type and the multiple types.

[0174] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: performing segmentation processing on the video data to be processed using a shot boundary detection technology to obtain a plurality of shot data; and determining target segmentation data from the plurality of shot data.

[0175] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: displaying a first recognition result to a target object; receiving feedback information from the target object based on the first recognition result; and triggering other segmented data when the feedback information indicates that the type indicated by the first recognition result is an invalid type.

[0176] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: obtaining target segmented data of the multimedia data to be processed, wherein the target segmented data is segmented data obtained after segmenting the multimedia data; identifying the type of the target segmented data to obtain a first recognition result; and correcting the first recognition result based on the type recognition results of other segmented data in the multimedia data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0177] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: obtaining target segmented data of the audio data to be processed, wherein the target segmented data is segmented data obtained after segmenting the audio data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result based on the types of other segmented data in the audio data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0178] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: obtaining target segmented data of the text data to be processed, wherein the target segmented data is segmented data obtained after segmenting the text data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result based on the types of other segmented data in the text data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0179] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: obtaining target segmented data of an image sequence to be processed, wherein the target segmented data is segmented data obtained after segmenting the image sequence data; identifying the type of the target segmented data to obtain a first recognition result of the target segmented data; and correcting the first recognition result based on the types of other segmented data in the image sequence data to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data.

[0180] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0181] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0183] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0186] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire target segmented data of the video data to be processed, wherein the target segmented data includes key frames; Identify the type of the key frame to obtain a first identification result; Determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; The first recognition result is modified according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data, Among them, correcting the first recognition result according to the other segmented data includes: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

2. The method according to claim 1, characterized in that The types include: scenes of the key frames; Before identifying the type of the key frame, the method further includes: Determine the scene to which the key frame belongs; Determining whether the scene type is a target type; When the scene type is the target type, it is determined that the scene of the key frame is allowed to be recognized; when the scene type is not the target type, it is determined that the scene of the key frame is rejected to be recognized.

3. The method according to claim 2, characterized in that Determining the scene to which the key frame belongs, including: Determine the proportion of the size of the target object in the picture corresponding to the key frame in the picture; Determining a value range to which the ratio belongs; The scene scale corresponding to the value range is determined, and the scene scale corresponding to the value range is used as the scene scale to which the key frame belongs.

4. The method according to claim 1, characterized in that Identify the type of the key frame, including: The image corresponding to the key frame is input into a neural network model for analysis to obtain the first recognition result, wherein the neural network model is trained using multiple groups of type annotation data.

5. The method according to claim 1, characterized in that Determining other segmented data in the to-be-processed video data includes: determining N segmented data in the to-be-processed video data that are located before the target segmented data, where N is a positive integer; Modifying the first recognition result according to the other segmented data includes: determining the types to which the N segmented data belong; and modifying the first recognition result according to the association relationship between the types of the N segmented data.

6. The method according to claim 5, characterized in that The first recognition result is modified according to the association relationship between the types of the N segmented data, including: Determining the reference type to which the target segmented data belongs according to the association relationship between the types of the N segmented data; The first recognition result is modified according to the reference type to which the target segment data belongs.

7. The method according to claim 6, characterized in that Determining the reference type to which the target segmented data belongs according to the association relationship between the types of the N segmented data includes: Determine the types of the N segmented data to obtain multiple types; determining a common type to which the plurality of types belong; The reference type to which the target segment data belongs is determined according to the subordinate relationship between the common type and the multiple types.

8. The method according to claim 1, characterized in that Obtaining target segment data of the video data to be processed includes: Using a shot boundary detection technology to perform shot division processing on the video data to be processed to obtain a plurality of shot data; The target segment data is determined from the plurality of shot data.

9. The method according to claim 1, characterized in that: Before determining other segmented data in the video data to be processed, the method further includes: Displaying the first recognition result to the target object; receiving feedback information fed back by the target object according to the first recognition result; When the feedback information indicates that the type indicated by the first recognition result is an invalid type, the other segmented data is triggered.

10. A data processing method, characterized in that: include: Acquire target segmented data of the multimedia data to be processed, wherein the target segmented data is segmented data obtained after segmenting the multimedia data; Identify the type of the target segmented data to obtain a first identification result; According to the type recognition results of other segmented data in the multimedia data, the first recognition result is modified to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data; Among them, the first recognition result is corrected according to the type recognition results of other segmented data in the multimedia data, including: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

11. A data processing method, characterized in that: include: Acquire target segmented data of the audio data to be processed, wherein the target segmented data is segmented data obtained after segmenting the audio data; Identify the type of the target segmented data to obtain a first identification result of the target segmented data; According to the type of other segmented data in the audio data, the first recognition result is modified to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data; Among them, the first recognition result is corrected according to the types of other segmented data in the audio data, including: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

12. A data processing method, characterized in that: include: Acquire target segmented data of the text data to be processed, wherein the target segmented data is segmented data obtained after segmenting the text data; Identify the type of the target segmented data to obtain a first identification result of the target segmented data; According to the type of other segmented data in the text data, the first recognition result is modified to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data; Among them, the first recognition result is corrected according to the types of other segmented data in the text data, including: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

13. A data processing method, characterized in that: include: Acquire target segmented data of the image sequence to be processed, wherein the target segmented data is segmented data obtained after segmenting the image sequence data; Identify the type of the target segmented data to obtain a first identification result of the target segmented data; According to the type of other segmented data in the image sequence data, the first recognition result is corrected to obtain a second recognition result, wherein the other segmented data is segmented data associated with the target segmented data; Among them, the first recognition result is corrected according to the types of other segmented data in the image sequence data, including: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

14. A data processing device, characterized in that: include: An acquisition module, used for acquiring target segmented data of the video data to be processed, wherein the target segmented data includes key frames; An identification module, used to identify the type of the key frame and obtain a first identification result; A determination module, configured to determine other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; The correction module is used to correct the first recognition result according to the other segmented data to obtain a second recognition result, including: determining the type to which the other segmented data belongs, determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs; wherein the second recognition result is used to indicate the type of the target segmented data.

15. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 13.

16. A computing device, characterized in that include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Acquire target segmented data of the video data to be processed, wherein the target segmented data includes key frames; Identify the type of the key frame to obtain a first identification result; Determining other segmented data in the video data to be processed, wherein the other segmented data is at least one segmented data in the video data to be processed except the target segmented data; Correcting the first recognition result according to the other segmented data to obtain a second recognition result, wherein the second recognition result is used to indicate the type of the target segmented data; Among them, correcting the first recognition result according to the other segmented data includes: determining the type to which the other segmented data belongs; determining the reference type to which the target segmented data belongs according to the association relationship between the types of the other segmented data, and correcting the first recognition result according to the reference type to which the target segmented data belongs.

Images