Method and apparatus for processing data

Abstract

The application discloses a data processing method and device. The method comprises the following steps: obtaining a video to be identified; and performing classification detection on the video to be identified by using a target classification model, and detecting at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video; wherein the target classification model at least comprises a same backbone network as an initial classification model and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing to identify the classification object meeting the default type from any video. The application solves the technical problem that in the prior art, when a new detection category is added, a detector needs to be retrained, thereby increasing deployment cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular, to a data processing method and device. BACKGROUND

[0002] A target detection algorithm can identify specific target entities in an image or video by detecting the image or video frame sequence, and then classify the image. The target detection algorithm can be applied to vehicle detection (e.g., identification and detection of various types of motor vehicles and non-motor vehicles such as cars, trucks, buses, motorcycles, electric vehicles, bicycles, etc.), commodity detection (e.g., detection of various types of commodities such as beauty, personal care, health, clothing, shoes, bags, digital, etc.), identification detection (e.g., detection of various types of identification such as station logo, trademark, road sign, etc.), and other fields. In the live video platform, new detection categories are often added according to needs to better control the live video. However, the target detector already online on the live video platform in the related art can only detect the existing categories in the training samples after training, and a new detector needs to be retrained for new categories. This method will result in the need for a new detector online on the live video platform if new detection categories are added to the live video, and with the continuous increase of detection categories, multiple detectors exist on the platform at the same time, greatly increasing the deployment cost.

[0003] At present, there is no effective solution to the problem of increasing deployment costs caused by the need to retrain a detector when adding new detection categories in the prior art. SUMMARY

[0004] The embodiments of the present application provide a data processing method and device to at least solve the technical problem of increasing deployment costs caused by the need to retrain a detector when adding new detection categories in the prior art.

[0005] According to an aspect of an embodiment of the present application, a data processing method is provided, comprising: obtaining a video to be identified; and performing classification detection on the video to be identified using a target classification model to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing identification of a classification object satisfying the default type from any video.

[0006] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: acquiring a live video to be identified collected in a live broadcast process; performing classification detection on the live video to be identified by using a target classification model, to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the live video; performing auditing on the classification object belonging to the default type and / or the classification object belonging to the incremental type, to obtain an auditing result; adding label information to the object reserved in the auditing result; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object belonging to the incremental type, and the initial classification model is a model allowing to identify the classification object satisfying the default type from any video.

[0007] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: acquiring a live video to be identified collected in a live broadcast process; performing classification detection on the live video to be identified by using a target classification model, to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the live video; performing auditing on the classification object belonging to the default type and / or the classification object belonging to the incremental type, to obtain an auditing result; adding label information to the object reserved in the auditing result; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object belonging to the incremental type, and the initial classification model is a model allowing to identify the classification object satisfying the default type from any video.

[0008] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: acquiring a live video to be identified collected in a live broadcast process; performing classification detection on the live video to be identified by using a target classification model, to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the live video; performing auditing on the classification object belonging to the default type and / or the classification object belonging to the incremental type, to obtain an auditing result; adding label information to the object reserved in the auditing result; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object belonging to the incremental type, and the initial classification model is a model allowing to identify the classification object satisfying the default type from any video.

[0009] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: obtaining courseware content to be identified in a teaching video, wherein different types of teaching content are shown in the courseware content; in response to a detection instruction sensed in an operation interface of the teaching video, calling a target classification model; and displaying a detection result of detecting the courseware content in the teaching video, wherein the courseware content is classified and detected by using the target classification model, and the detection result includes teaching content belonging to a default type and at least one type of teaching content belonging to an incremental type in the courseware content; wherein the target classification model at least includes a same backbone network as an initial classification model and a model for identifying the teaching content belonging to the incremental type, and the initial classification model is a model allowing to identify the teaching content belonging to the default type from any video.

[0010] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: obtaining courseware content to be identified in a teaching video, wherein different types of teaching content are shown in the courseware content; in response to a detection instruction sensed in an operation interface of the teaching video, calling a target classification model; and displaying a detection result of detecting the courseware content in the teaching video, wherein the courseware content is classified and detected by using the target classification model, and the detection result includes teaching content belonging to a default type and at least one type of teaching content belonging to an incremental type in the courseware content; wherein the target classification model at least includes a same backbone network as an initial classification model and a model for identifying the teaching content belonging to the incremental type, and the initial classification model is a model allowing to identify the teaching content belonging to the default type from any video.

[0011] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: obtaining courseware content to be identified in a teaching video, wherein different types of teaching content are shown in the courseware content; in response to a detection instruction sensed in an operation interface of the teaching video, calling a target classification model; and displaying a detection result of detecting the courseware content in the teaching video, wherein the courseware content is classified and detected by using the target classification model, and the detection result includes teaching content belonging to a default type and at least one type of teaching content belonging to an incremental type in the courseware content; wherein the target classification model at least includes a same backbone network as an initial classification model and a model for identifying the teaching content belonging to the incremental type, and the initial classification model is a model allowing to identify the teaching content belonging to the default type from any video.

[0012] According to another aspect of the embodiments of the present application, a data processing method is provided, comprising: obtaining a video recognition parameter and a live video to be recognized; performing optimization processing on an initial classification model based on the live video recognition parameter to obtain a target classification model, wherein the initial classification model is a model allowing to recognize an object meeting a first classification from any video, and the video recognition parameter comprises identification information representing an object meeting a second classification; training the target classification model using sample data, wherein the target classification model uses the same backbone network as the initial classification model, and the sample data comprises: sample images and labeling information for labeling the object meeting the second classification in the sample images; and performing classification detection on the live video based on the trained target classification model to obtain the object meeting the first classification and / or the object meeting the second classification in the live video.

[0013] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a video acquisition module configured to obtain a video to be recognized; and a classification module configured to perform classification detection on the video to be recognized using a target classification model to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video, wherein the target classification model comprises at least a same backbone network as an initial classification model and a model for recognizing the classification object belonging to the incremental type, and the initial classification model is a model allowing to recognize the classification object meeting the default type from any video.

[0014] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a collection module configured to obtain a live video to be recognized collected in a live process; a detection module configured to perform classification detection on the live video to be recognized using a target classification model to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the live video; an auditing module configured to audit the classification object belonging to the default type and / or the classification object belonging to the incremental type to obtain an auditing result; and an adding module configured to add label information to the object reserved in the auditing result, wherein the target classification model comprises at least a same backbone network as an initial classification model and a model for recognizing the classification object belonging to the incremental type, and the initial classification model is a model allowing to recognize the classification object meeting the default type from any video.

[0015] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, comprising a stored program, wherein the program controls a device where the computer readable storage medium is located to perform the data processing method of any of the above when the program is running.

[0016] According to another aspect of the embodiments of the present application, a processor is also provided, which is used to run a program, wherein the program performs the data processing method of any one of the above aspects when running.

[0017] In the embodiments of the present application, by acquiring a video to be recognized, a target classification model is used to perform classification detection on the video to be recognized, at least a classification object belonging to a default type and at least one classification object belonging to an incremental type are detected from the video, wherein the target classification model at least includes a same backbone network as the initial classification model and a model for recognizing the classification object of the incremental type, the target classification model which can be obtained by optimizing the structure and parameters based on the initial classification model is realized, the expansion of the detection categories is realized, a new classification model does not need to be redesigned, and then the dynamic expansion of the detection categories can be performed by using only one classification model for an image field on a live video platform, and the problem that a detector needs to be retrained when a new detection category is added in the prior art, resulting in an increase in deployment cost, is solved. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computing device for implementing the data processing method;

[0020] Figure 2 is a flowchart of a data processing method according to Embodiment 1 of the present application;

[0021] Figure 3 is a system architecture diagram of an optional data processing method according to Embodiment 1 of the present application;

[0022] Figure 4 is a flowchart of a data processing method according to Embodiment 2 of the present application;

[0023] Figure 5 is a flowchart of a data processing method according to Embodiment 3 of the present application;

[0024] Figure 6 is a flowchart of a data processing method according to Embodiment 4 of the present application;

[0025] Figure 7 is a flowchart of a data processing method according to Embodiment 5 of the present application;

[0026] Figure 8 is a flowchart of a data processing method according to Embodiment 6 of the present application;

[0027] Figure 9 is a schematic diagram of a data processing device according to an embodiment 7 of the present application;

[0028] Figure 10 is a schematic diagram of a data processing device according to an embodiment 8 of the present application;

[0029] Figure 11 is a schematic diagram of a data processing device according to an embodiment 9 of the present application;

[0030] Figure 12 is a schematic diagram of a data processing device according to an embodiment 10 of the present application;

[0031] Figure 13 is a schematic diagram of a data processing device according to an embodiment 11 of the present application;

[0032] Figure 14 is a schematic diagram of a data processing device according to an embodiment 12 of the present application;

[0033] Figure 15 is a schematic diagram of a data processing device according to an embodiment 13 of the present application;

[0034] Figure 16 is a schematic diagram of a data processing device according to an embodiment 14 of the present application;

[0035] Figure 17 is a schematic diagram of a data processing device according to an embodiment 15 of the present application;

[0036] Figure 18 is a schematic diagram of a data processing device according to an embodiment 16 of the present application;

[0037] Figure 19 is a structural block diagram of a computer terminal according to an embodiment 17 of the present application. DETAILED DESCRIPTION

[0038] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without making creative labor should belong to the protection scope of the present application.

[0039] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described drawings are intended to distinguish similar objects and are not necessarily intended to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the application described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a list of steps or units can not necessarily be limited to those steps or units that are clearly listed, but can include other steps or units that are not expressly listed or inherent to such process, method, product, or apparatus.

[0040] It should be noted that the schemes of the embodiments of the present application can be combined with each other without conflict, and the new schemes generated by the combination are also within the scope of protection of the present application.

[0041] Embodiment 1

[0042] According to the embodiments of the present application, a data processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0043] The method embodiment provided by the embodiment of the present application can be executed in a mobile terminal, a computing device or a similar computing device. Figure 1 A hardware structure block diagram of a computing device (or mobile device) for implementing the data processing method is shown. As shown in Figure 1 The computing device 10 (or mobile device 10) can include one or more processors 102 (the processor 102 can include but is not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 104 for storing data, and a transmission module 106 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computing device 10 can include more or less components than those shown in Figure 1 or have a different configuration than that shown in Figure 1 .

[0044] It should be noted that the one or more processors 102 and / or other data processing circuitry described above can be generally referred to herein as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any of the other elements of the computing device 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuitry serves as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0045] The memory 104 can be used to store software programs and modules of application software, as well as program instructions / data storage means corresponding to the data processing method of the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e., implements the vulnerability detection method of the application program described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely disposed relative to the processor 102, which can be connected to the computing device 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission module 106 is configured to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computing device 10. In one example, the transmission module 106 includes a network adapter (NIC), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0047] The display can be, for example, a touch screen type liquid crystal display (LCD), which can enable a user to interact with the user interface of the computing device 10 (or mobile device).

[0048] In the above operating environment, the present application provides a data processing method as shown in Figure 2 Figure 2 is a flowchart of a data processing method according to Embodiment 1 of the present application, as shown in Figure 2 The method includes the following steps:

[0049] In step S201, a video to be recognized is acquired. ​

[0050] The video to be identified is a video that needs to be detected and classified. The video to be identified can be a video of any theme or application scenario, including but not limited to a live video obtained on a live platform, a traffic video of a traffic system, a teaching video in the field of education, a medical examination video in the field of medicine, and the like.

[0051] For example, in the scenario of intelligent traffic, the video can be a traffic video; in the entertainment scenario, the video can be a video of a film and television; in the intelligent teaching scenario, the video can be a teaching video; and in the intelligent medical scenario, the video can be a medical image video.

[0052] In step S202, a target classification model is used to perform classification detection on the video to be identified, and at least a classification object belonging to a default type and at least one classification object belonging to an incremental type are detected from the video; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing to identify the classification object of the default type from any video.

[0053] Specifically, the classification object of the default type is a category that can be classified and detected by the initial classification model, and the classification object of the incremental type is a category that cannot be identified by the initial classification model, i.e., a new category that can be identified after the initial classification model is optimized and trained.

[0054] The classification object of the default type and the classification object of the incremental type can each include multiple categories, i.e., the initial classification model can be used to implement classification detection of multiple categories. The classification object of the default type and the classification object of the incremental type are different. For example, the classification object of the default type can be a product A in a video image, and the classification object of the incremental type can be a product B in the image.

[0055] The backbone network is a feature extraction network for extracting features of a video image. The target classification model uses the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model, for example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network, which is used to perform classification detection on the features extracted by the backbone network.

[0056] The target classification model can detect the default type of classification object and the incremental type of classification object. In actual scenarios, there are cases where the live video to be identified contains only the default type of classification object or only the incremental type of classification object. Therefore, the target classification model can detect any one or several of the default type of classification object and the incremental type of classification object in the live video.

[0057] In an optional embodiment, in a live video platform, the video to be identified is a large amount of live video on the live video platform, and the initial classification model is an initial classification model that has been put online on the live video platform and can be used to detect the existing category M. According to the user's demand, the video recognition parameter is set to category N, so as to propose the need to detect category N on the live video platform. However, the current initial classification model cannot detect category N. Therefore, according to the structure of the initial classification model, a target classification model with the same backbone network as the initial classification model is constructed, and the target classification model is trained using sample images with label information of category N. The trained target classification model can replace the initial classification model to detect category M and category N on the live video platform, thereby realizing the expansion of the detection category on the live video platform without increasing the number of classification models, and saving the deployment cost of the classification model on the platform.

[0058] In this embodiment, the video to be identified is obtained, and the target classification model is used to perform classification detection on the video to be identified, so as to detect at least a classification object belonging to the default type and at least one classification object belonging to the incremental type from the video. The target classification model at least includes the same backbone network as the initial classification model and a model for identifying the classification object of the incremental type. The target classification model can be obtained by optimizing the structure and parameters of the initial classification model, thereby realizing the expansion of the detection category, without the need to redesign a new classification model. Furthermore, the same classification model can be used for dynamic expansion of the detection category in an image field on the live video platform, thereby solving the problem of increasing the deployment cost caused by the need to retrain a detector when a new detection category is added in the prior art.

[0059] As an optional embodiment, before the target classification model is used to perform classification detection on the video to be identified, the above method further includes:

[0060] In step S2011, sample data and video recognition parameters are obtained.

[0061] The video recognition parameter is a category that needs to be identified in the video to be identified, which can be set according to the user's demand. For example, after obtaining a live video of an e-commerce platform, the video recognition parameter can be set as commodity A and commodity B, and then the classification detection model is used to identify the two categories of commodity A and commodity B in the live video to be identified.

[0062] The sample data can be labeled based on a video obtained from the Internet, and used to train the target classification model, so that the target classification model can classify the incremental type of classification object

[0063] In step S2012, the initial classification model is optimized based on the video recognition parameter to obtain the target classification model, wherein the video recognition parameter includes identification information representing the incremental type of classification object.

[0064] The video recognition parameter can include identification information of the default type of classification object and identification information of the incremental type of classification object at the same time, or only include identification information representing the incremental type of classification object. The above identification information can be the name of the incremental type of classification object.

[0065] The optimization of the initial classification model can be to construct the target classification model based on the structure and parameters of the initial classification model, and to improve the structure and parameters of the initial classification model, so that the target classification model can meet the requirements of classifying and detecting the live video according to the video recognition parameter. For example, after obtaining a live video, the video recognition parameter can be set as commodity A and commodity B, and the initial classification model can only recognize commodity A but cannot recognize commodity B. The structure and parameters of the initial classification model are optimized to obtain the target classification model, and the target classification model can detect commodity B after training.

[0066] In an optional embodiment, the optimization of the initial classification model to obtain the target classification model can be that the backbone network in the target classification model reuses the backbone network of the initial classification model, and a new head network is added after the backbone network in the target classification model. The network parameters of the reused backbone network of the initial classification model are used to train the initial classification model as a pre-training network. The newly added head network in the target classification model can be used to detect the features extracted by the backbone network of the target classification model. After the training of the target classification model is completed, the newly added head network can be used to detect the features extracted by the backbone network of the target classification model. The default type of classification object and the incremental type of classification object. The network parameters of the newly added head network in the target classification model can be the same as or different from the head network of the initial classification model in the initial setting. The number of nodes in the newly added head network in the target classification model should be equal to the number of categories in the newly added incremental type of classification object.

[0067] In step S2013, the target classification model is trained using sample data, wherein the sample data includes sample images and label information for labeling the incremental type of classification object in the sample images.

[0068] The marking information of the classification object belonging to the incremental type in the sample image can be obtained by manual marking. After determining the classification object of the incremental type as the newly added classification, the above-mentioned marking information is obtained by manually marking the sample image. For example, when the video to be recognized is a live video of an e-commerce platform, if it is necessary to add detection of the commodity B, it is determined that the commodity B is the classification object of the incremental type, and the "commodity B" is marked in the sample image by a frame.

[0069] The target classification model is trained and learned using the marking information of the sample image on the classification object of the incremental type, so that the target classification model can classify and detect the classification object of the incremental type in the image. In an optional embodiment, the sample image is input into an initial classification model, and the initial classification model outputs a detection result of the classification object of the default type in the sample image. The target classification model learns the classification object of the default type based on the distillation knowledge of the classification object of the default type, so that the target classification model can realize classification and detection of the classification object of the default type and the classification object of the incremental type.

[0070] The above-mentioned sample data includes sample images and marking information of classification results of the sample images on the classification object of the incremental type. The sample images and the marking information of the sample images can be used to train a classification model based on a target detection algorithm, so that the classification model learns the newly added classification object of the incremental type. In an optional embodiment, the sample data includes only sample images and marking information of classification results of the sample images on the classification object of the incremental type. Since the classification object of the default type is a classification that can be recognized by the initial classification model, it is not necessary to mark the classification object of the default type in the sample image. The marking information in the sample data is only the marking information of the classification object of the incremental type, which can reduce the data amount of the sample data, so as to reduce the data processing amount of the computing device.

[0071] As an optional embodiment, after at least the classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, the above-mentioned method further includes: displaying the recognized classification object of the default type and the classification object of the incremental type, including any one or more of the following: marking the classification object of the default type and the classification object of the incremental type in each frame of the video by different marking frames; displaying the number of times of appearance of the classification object of the incremental type in a preset time period.

[0072] In the embodiment of the e-commerce live broadcast platform, the classification object of the default type is commodity A, the classification object of the incremental type is commodity B, and after the target classification model detects commodity A or commodity B from the live broadcast video, the target classification model can frame the detected commodity A or commodity B in each frame of the video with a marking box (for example, a square box), so that the user pays attention to the position of commodity A or commodity B in each frame of the video. Different marking boxes can be set for different classification objects, such as different colors or different border lines.

[0073] The number of times that the classification object of the incremental type appears in the preset time period is displayed, which can be the number of times of appearance in a certain video. By displaying the number of times that the classification object of the incremental type appears in the preset time period, the detection accuracy of the target classification model for the classification object of the incremental type can be determined. The above-mentioned preset time can be any time period in the video, which can be set according to the user's demand. For example, the preset time can be set as the time corresponding to the first frame to the tenth frame in the video. If the classification object of the incremental type appears in one frame of image, it is considered to appear once. The number of times that the classification object of the incremental type appears in 10 frames of images is displayed.

[0074] As an optional embodiment, after at least the classification object of the default type and at least one classification object of the incremental type are detected from the video, the above-mentioned method further includes any one or more of the following: in the case that the number of times that the classification object of the incremental type appears in the preset time period is greater than a preset number of times, increasing the sample data used for training the target classification model; obtaining a category with a recognition accuracy lower than a preset threshold, and labeling the category with a recognition accuracy lower than the preset threshold.

[0075] The above-mentioned preset number of times is a reasonable number of times that the classification object of the incremental type appears in the preset time period. If the number of times of appearance is greater than the preset number of times, it can be considered that the detection accuracy of the target classification model for the classification object of the incremental type is low, which leads to the misrecognition of other categories of classification objects as the classification object of the incremental type. Therefore, more sample data is needed to train the target classification model to improve the detection accuracy of the target classification model for the classification object of the incremental type. For example, the target classification model detects commodity B in the first frame to the 50th frame of the video. The preset number of times can be 50, that is, commodity B appears once in each frame. If the number of times of appearance obtained by detection is greater than 50 times, it can be considered that other categories of commodities in a certain frame of image are mistaken for commodity B.

[0076] The recognition accuracy can be determined according to the number of times of appearance of the classification object detected by the target classification model in a preset time period and the number of times of actual appearance of the classification object in each frame image of the video. By labeling the categories with recognition accuracy lower than a preset threshold, the user can pay attention to the classification object prone to recognition errors, and then train the target classification model to improve the accuracy of detection of the classification object. It should be noted that the classification object identified according to the accuracy includes the classification object of the default type and the classification object of the incremental type, and any classification object with low recognition accuracy will be identified. The above-mentioned preset threshold can be determined according to the user's demand for detection accuracy, which is not limited here.

[0077] As an optional embodiment, after at least one classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, the method further comprises:

[0078] receiving correction information, wherein the correction information is used to determine whether the recognition result is accurate, and the recognition result includes the classification object of the default type and the classification object of the incremental type; in the case that the correction information indicates that the recognition result is incorrect, receiving the correction information and correcting the recognition result based on the correction information; and displaying the corrected recognition result.

[0079] The above-mentioned correction information can be determined by a person according to the classification object of the default type and the classification object of the incremental type displayed on each frame image of the video. In the case that the correction information indicates that the recognition result is incorrect, the correction information is inputted by the person to display the correct recognition result.

[0080] For example, in the embodiment of the e-commerce live broadcast platform, after classification detection, the goods A in the video are framed by the label frame. Whether the content framed by the label frame is goods A can be determined by a person. If the framed content is actually goods B, the correction information indicating that the recognition result is incorrect is sent. Further, the identification information of the framed content of the label frame can be modified as "goods B" as the correction information, and goods B and its identification information are displayed in the video. Alternatively, a label frame can be added to the position of goods A in the image of the video and marked as "goods A" as the correction information to correct the recognition result, so that goods A and its identification information are accurately displayed in the video. The above-mentioned correction information can also include the correction information of goods A and goods B at the same time, so as to accurately display goods A and its identification information, and goods B and its identification information.

[0081] As an optional embodiment, Figure 3 is a system architecture diagram of an optional data processing method according to an embodiment of the present application, like Figure 3As shown, the target classification model comprises a second backbone network and a second head network, and the second head network comprises a second classification branch 321 and a second regression branch 322.

[0082] The second backbone network is configured to perform feature extraction on the video to be recognized, and the second head network is configured to perform detection on the features extracted by the first backbone network. After the target classification model is trained, the second head network is configured to perform detection on the features extracted by the first backbone network to detect the default type of classification object and the incremental type of classification object.

[0083] As an optional embodiment, the above method further comprises obtaining the target classification model, and the obtaining of the target classification model comprises the following steps S2021-S2024:

[0084] S2021, obtaining an initial detection model and sample data, wherein the initial detection model comprises a first backbone network configured to perform feature extraction and a first head network configured to perform classification detection of the default type of classification object, and the sample data comprises sample images and label information of the second type of classification object in the sample images.

[0085] The initial classification model is a classification model with determined parameters. Through the initial classification model, the default type of classification object in the image or the image with the default type of classification object can be recognized and identified, but the incremental type of classification object in the image or the image with the incremental type of classification object cannot be recognized and identified. Correspondingly, the first backbone network and the first head network in the initial classification model are neural networks with determined parameters. Any image obtained is extracted by the first backbone network to obtain features, and the first head network is used for classification to determine whether the image contains the features corresponding to the default type of classification object. For example, for target detection of live video of an e-commerce live broadcast platform, the default type of classification object can be two categories of goods A and goods B, and the incremental type of classification object can be goods C. Any image obtained is input into the first backbone network to extract features, and the first head network is used for classification detection to determine whether the image contains goods A and goods B. However, the first head network cannot detect goods C which belongs to the incremental type of classification object.

[0086] S2022, performing reuse processing on network parameters of the first backbone network to generate a second backbone network.

[0087] It should be noted that, in the case where the second backbone network has not been trained, the network parameters of the second backbone network are the same as the network parameters of the first backbone network. For example, the first backbone network can be a 3*3 encoder-decoder structure, and the second backbone network can also be a 3*3 encoder-decoder structure. The first backbone network and the second backbone network have the same up-sampling layer and down-sampling layer.

[0088] In the above scheme, the network parameters of the first backbone network are reused to use the first backbone network as a pre-training network and train the first backbone network based on the pre-training network.

[0089] S2023, a second head network is added at the output end of the second backbone network to obtain a target classification model.

[0090] The target classification model is a classification model that needs to be trained, and the target classification model includes a second backbone network with the same network parameters as the first backbone network. The network parameters of the second head network can be the same as or different from the first head network when initially set. The number of nodes in the second head network should be equal to the number of categories in the incremental type of classification object.

[0091] S2024, training the target classification model based on sample data, wherein the target classification model is used for detecting the default type of classification object and the incremental type of classification object.

[0092] After the network parameters of the target classification model are determined, the target classification model is trained and learned using the label information containing the classification results of the sample images on the incremental type of classification object, so that the target classification model can classify and detect the incremental type of classification object in the image. In an optional embodiment, the sample image is input into the initial classification model, the initial classification model outputs the detection result of the default type of classification object in the sample image, the target classification model learns the default type of classification object based on the distillation knowledge of the default type of classification object, and then the target classification model can realize classification and detection of the default type of classification object and the incremental type of classification object.

[0093] It should be noted that when the target classification model is trained, the target classification model can replace the original initial classification model, and the trained target classification model can detect the default type of classification object and the incremental type of classification object in the original image field, so that only one classification model is used in the original image field to realize detection of the original categories and the new categories. When the image field needs to add new categories again, the above default type of classification object and the incremental type of classification object learned by the target classification model can be combined as new default type of classification object, and the new categories that need to be added again can be used as new incremental type of classification object. The target classification model that has been trained is used as a new initial classification model, and the above steps S2021-S2024 are repeated to obtain a new target classification model. Through the above scheme, the classification model with the same network parameter backbone network can be used to learn the new categories based on the sample data with only the new category label information, so as to dynamically expand the detection categories in the same image field, so that the same classification model can learn new categories without forgetting the original categories.

[0094] In an optional embodiment, for the target detection algorithm of the commodity type of the e-commerce platform commodity live video, the classification objects of the default type can be two categories of commodity A and commodity B, and the initial classification model is a target detector X that has been online and can be used to detect the two categories of commodity A and commodity B. When further detection of commodity C is needed, commodity C is taken as a classification object of an incremental type, a live video of the e-commerce platform containing commodity C is taken as a sample image, and after marking commodity C in the sample image, the sample image and the sample image are taken as sample data together. The network parameters of the first backbone network of the target detector X are reused to obtain a target detector Y as a target classification model, and the target detector Y is trained using the labeled information of commodity C and the sample image, so that the target detector Y learns the category of "commodity C". After training is completed, the target detector Y can detect three categories of commodity A, commodity B and commodity C, and the target detector Y can be online to replace the target detector X as a target detector. When further detection of commodity D is needed, commodity A, commodity B and commodity C are taken as the above-mentioned classification objects of the default type, and commodity D is taken as the above-mentioned classification object of the incremental type. The sample data is obtained by labeling commodity D in the sample image, the target detector Y is taken as the above-mentioned initial classification model, the network parameters of the first backbone network of the target detector Y are reused to obtain a target detector Z as a target classification model, and the target detector Z is trained according to the above-mentioned training method to learn the category of "commodity D". The target detector Z is online to replace the target detector Y as the target detector of the commodity field. By analogy, in the same image field, only one target detector is used, and through training and learning of the new category, a target detector capable of detecting the original category and the new category at the same time can be obtained, thereby avoiding the problem of increasing the online deployment cost caused by using multiple target detectors.

[0095] As an optional embodiment, Figure 3 is a system architecture diagram of an optional data processing method according to an embodiment of the present application, as Figure 3As shown, the first head network includes a first classification branch 311 for performing classification detection of a default type of classification object and a first regression branch 312 for performing bounding box regression. The target classification model is trained based on sample data so that the target classification model is used to perform detection of the default type of classification object and an incremental type of classification object, including: inputting the sample data into an initial detection model to obtain a first classification result output by the first classification branch 311 and a first regression result output by the first regression branch 312; inputting the sample data into the target classification model to obtain a second classification result output by a second classification branch 321 and a second regression result output by a second regression branch 322; determining a target loss function based on the first classification result, the second classification result, the first regression result, and the second regression result; and optimizing the target classification model based on the target loss function.

[0096] The first classification result is used to represent whether the sample image of the sample data contains the default type of classification object. The first regression result is used to represent the accuracy of the positioning (for example, the bounding box or coordinates of the default type of classification object in the image) of the default type of classification object in the sample image. It should be noted that the initial classification model is an existing classification model that can detect the default type of classification object. The first classification result and the first regression result can be detection results output by the initial classification model for the default type of classification object that has been learned, and can be used as distilled knowledge to be transferred to the target classification model to be trained, so that the target classification model learns the default type of classification object.

[0097] The second classification result and the second regression result can be used for the target classification model to identify the incremental type of classification object. Specifically, the second classification result and the second regression result can be used to determine a loss function for training the target classification model based on the sample data to learn the incremental type of classification object. In an optional embodiment, as shown, Figure 3 As shown, the sample data 30 includes sample images and label information of the sample images on the incremental type of classification object. After the sample images are extracted by the second backbone network 32, the second classification branch 321 performs classification detection on the extracted features to output the second classification result, and the second regression branch 322 performs bounding box regression on the extracted features to output the second regression result. Then, the second classification result and the second regression result are calculated with the label information of the sample images on the incremental type of classification object to obtain a second classification loss function and a second regression loss function.

[0098] The target loss function is determined based on the first classification result, the second classification result, the first regression result and the second regression result. The first classification loss function, the second classification loss function, the first regression loss function and the second regression loss function can be determined respectively based on the first classification result, the second classification result, the first regression result and the second regression result, and the target loss function is obtained by weighting the above four loss functions. The target loss function can also be determined by weighting the first classification result, the second classification result, the first regression result and the second regression result. In the scheme, the target classification model is trained by using the target loss function obtained based on the first classification result, the second classification result, the first regression result and the second regression result, so that the target classification model learns the classification objects of the default type and the classification objects of the incremental type at the same time.

[0099] In an optional implementation, the first classification branch, the second classification branch, the first regression branch and the second regression branch can use any one of an AlexNet network, a VGG-16 network and a ResNet network.

[0100] In an optional embodiment, before the sample data is input into the initial detection model to obtain the first classification result output by the first classification branch 311 and the first regression result output by the first regression branch 312, the method further includes: fixing the network parameters of the initial classification model.

[0101] It should be noted that when the sample data containing only the classification object label information of the incremental type is input into the initial classification model, the initial classification model is the original classification model that has learned the classification objects of the default type. The network parameters of the initial classification model need to be kept unchanged to avoid the initial classification model learning the classification objects of the incremental type and changing the network parameters, and then causing the initial classification model to forget the learned classification objects of the default type. Fixing the network parameters of the initial classification model can make the initial classification model output the distillation knowledge of the classification objects of the default type based on the input sample data, so as to make the target classification model learn and realize the classification detection of the classification objects of the default type.

[0102] As an optional embodiment, as shown in Figure 3 The target loss function is determined based on the first classification result, the second classification result, the first regression result and the second regression result. The first classification loss function, the second classification loss function, the first regression loss function and the second regression loss function can be determined respectively based on the first classification result, the second classification result, the first regression result and the second regression result, and the target loss function is obtained by weighting the above four loss functions. The target loss function can also be determined by weighting the first classification result, the second classification result, the first regression result and the second regression result. In the scheme, the target classification model is trained by using the target loss function obtained based on the first classification result, the second classification result, the first regression result and the second regression result, so that the target classification model learns the classification objects of the default type and the classification objects of the incremental type at the same time.

[0103] The classification loss 34 and the regression loss 35 are respectively loss functions determined based on the second classification result and the second regression result, and can be used to train the target classification model to learn the incremental type of classification object. The classification loss 34 is used to represent the deviation of the second classification result output by the sample image through the second classification network from the classification in the label information of the sample image on the incremental type of classification object in the sample data, and the regression loss 35 is used to represent the deviation of the second regression result output by the sample image through the second regression network from the positioning in the label information of the sample image on the incremental type of classification object in the sample data. Training the target classification model using the classification loss 34 and the regression loss 35 can improve the recognition accuracy of the target classification model on the incremental type of classification object.

[0104] In an optional embodiment, the classification loss 34 can be determined by a Focal loss function, and the regression loss 35 can be determined by a GIoU function.

[0105] The classification distillation loss 33 is a loss function based on the first classification result and the second classification result, and the regression distillation loss 36 is a loss function based on the first regression result and the second regression result. The classification distillation loss 33 and the regression distillation loss 36 can represent the deviation of the output of the target classification model from the first classification result and the first regression result as distillation knowledge. Training the target classification model using the classification distillation loss 33 and the regression distillation loss 36 can enable the target classification model to achieve classification detection of the default type of classification object through learning.

[0106] In an optional embodiment, the classification distillation loss 33 can be determined by an L2 function, and the regression distillation loss 36 can be determined by a GIoU function.

[0107] It should be noted that the initial classification model can be used as a teacher network, and the default type of classification object learned by the initial classification model can be used as distillation knowledge to be passed to the target classification model. Training the target classification model based on the distillation knowledge of the default type of classification object and the loss function constructed by the second classification result and the second regression structure output by the target classification model from the sample data can enable the target classification model to achieve classification detection of the default type of classification object through learning from the sample data even if the sample data does not contain label information of the default type of classification object.

[0108] In an optional embodiment, for the target detection of the type of goods in the live video of the goods of the e-commerce platform, the default type of classification object can be two categories of goods A and goods B, and the sample data can include the label information of the goods A and the goods B. Figure 3As shown, the existing initial classification model includes a first backbone network 31, a first classification branch 311 and a first regression branch 312, and the initial classification model can detect two categories of commodity A and commodity B. For example, inputting an image with two categories of commodity A and commodity B into the first backbone network 31 can extract features of the image, and the features can be used by the first classification branch 311 to determine whether the image contains two categories of commodity A and commodity B, and the second regression branch 312 can output the accuracy of the bounding box position of commodity A and commodity B on the image. When a new "commodity C" is added as an incremental type of classification object, a target classification model including a second backbone network 32, a second classification branch 321 and a second regression branch 322 can be constructed, wherein the second backbone network 32 reuses the network parameters of the first backbone network 31. Sample data 30 containing tasks is obtained, and the sample data 30 contains sample images and label information about commodity C on the sample images (for example, commodity C is boxed by a bounding box). The sample data 30 is input into the first backbone network 31 and the second backbone network 32 respectively for feature extraction, the first classification branch 311 outputs classification distillation knowledge of the default type of classification object, and the first regression branch 312 outputs regression distillation knowledge of commodity A and commodity B. The second classification branch 321 classifies the features of the sample image to output a second classification result, and based on the second classification result and the classification distillation knowledge, a classification distillation loss function 33 for training the target classification model for commodity A and commodity B is determined. The second classification result is compared with the sample data to obtain a classification loss function 34 for training the target classification model for commodity C. Correspondingly, the second regression branch 322 performs bounding box regression on the features of the sample image to output a second regression result, and the second regression result is compared with the sample data to obtain a regression loss function 36 for regression training of the target classification model for commodity C. Based on the second regression result and the regression distillation knowledge, a regression distillation loss function 35 for regression training of the target classification model for commodity A and commodity B is determined. The obtained classification loss function 34, regression distillation loss function 35, classification distillation loss function 33 and regression loss function 36 are weighted and added to construct a target loss function, and the target classification model is trained using the target loss function, so that the target classification model can detect commodity A, commodity B and commodity C at the same time.

[0109] As an optional embodiment, the sample data is input into the initial detection model to obtain a first classification result output by the first classification branch and a first regression result output by the first regression branch, including: selecting a distillation node from output nodes of the first backbone network according to a confidence; the first classification branch outputs the first classification result based on feature information output by the distillation node; and the first regression branch outputs the first regression result based on the feature information output by the distillation node.

[0110] Since the sample data does not contain the label information of the default type of the classified object, the accuracy of the output result of the initial classification model cannot be represented by the loss function after the sample data is input into the initial classification model. Therefore, the output nodes of the first backbone network are screened by the confidence, and the nodes with high confidence are selected as the distillation nodes, so that the distillation knowledge of the output of the initial classification model is the detection result of the classified object of the default type.

[0111] The feature map output by the distillation node is taken as the positive sample of the first classification network and the first regression network, and the feature map output by the non-distillation node is taken as the negative sample of the first classification network and the first regression network. The classification and regression are performed based on the positive sample to obtain the first classification result and the first regression result.

[0112] In an optional embodiment, the first backbone network has a plurality of layers of convolutional neural networks, each layer of convolutional neural network includes a plurality of output nodes, and the distillation nodes are selected from the output nodes of the first backbone network according to the confidence, including: determining that the first N output nodes in the order of descending confidence are the distillation nodes.

[0113] The confidence is used to represent the reliability of the output node of the first backbone network in the detection result of the classified object of the default type, and the distillation node can be a node with a more accurate detection result selected from the output nodes of the first backbone network. For example, the top N nodes in the order of descending confidence are selected as the distillation nodes.

[0114] It should be noted that the total output nodes of the first backbone network are understood as the sum of the nodes in each convolutional layer. For example, the first backbone network is a neural network with three layers of convolutional layers, a sample image of 1024*1024 is input into the first backbone network, a feature map of 512*512 is output after feature extraction of the first layer network, a feature map of 256*256 is output after feature extraction of the second layer, and a feature map of 128*128 is output after feature extraction of the second layer. The first classification network and the first regression network process each pixel point in each feature map to generate the confidence and the regression bounding box value of each pixel point in each feature map. In the above example, there are 512*512+256*256+128*128 confidences, and the total 344064 output nodes are sorted according to the confidence, wherein the first N output nodes are the distillation nodes.

[0115] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0117] Example 2

[0118] According to an embodiment of the present invention, an embodiment of a data processing method is also provided. Figure 4 This is a flowchart of a data processing method according to Embodiment 2 of this application, as follows: Figure 4 As shown, the processing method includes the following steps:

[0119] Step S401: Obtain the live video to be identified captured during the live broadcast.

[0120] The live stream videos to be identified can be those that require detection and classification.

[0121] In the live video review scenario, the live video to be identified can be a live video being broadcast on a live video platform. The above-mentioned collection process can be to collect the live video before it is distributed to the user's end, classify and detect the collected live video, and then review the content of the live video to determine whether the live video being broadcast involves any illegal content categories, and then take corresponding pre-processing measures to prevent live videos containing illegal content from spreading on the network platform.

[0122] At step S402, the target classification model is used to perform classification detection on the live video to be recognized, and at least a classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the live video. The target classification model at least includes the same backbone network as the initial classification model, and a model for identifying classification objects of the incremental type. The initial classification model is a model that allows classification objects meeting the default type to be identified from any video.

[0123] Specifically, the classification object of the default classification is a category that can be classified and detected by the initial classification model, and the classification object of the incremental type is a category that cannot be identified by the initial classification model, i.e., a new category that can be identified after the initial classification model is optimized and trained. For example, in the live video review scenario, the classification object of the default classification can be a commodity A that can be identified by the live video platform. The live video platform can identify the commodity A in the live video collected during the live process. At this time, the live video platform cannot identify the commodity B, and the commodity B is the classification object of the incremental type. By training and optimizing the initial classification model already existing in the live video platform, the target classification model is obtained, so that the live video platform can identify the commodities A and B.

[0124] It should be noted that the backbone network is a feature extraction network for extracting features of a live video image. The target classification model uses the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network to perform classification detection on the features extracted by the backbone network.

[0125] At step S403, the classification object belonging to the default type and / or the classification object belonging to the incremental type is reviewed to obtain a review result.

[0126] The above review process can be to determine whether the live video containing the classification object belonging to the default classification and / or the classification object belonging to the incremental type is a violation video when the classification object belonging to the default classification and / or the classification object belonging to the incremental type is detected in the live video, to obtain the review result.

[0127] For example, in a live video platform of an e-commerce, the classification object belonging to the default classification can be a commodity A, and the classification object belonging to the incremental type can be a commodity B. Based on the above trained target classification model, the live video is recognized and detected, and the live video containing the commodity A or the commodity B is recognized. The live video containing the commodity A or the commodity B is output in the live video review interface, and the reviewer confirms whether the live video is illegal or not, or further judges the region where the recognized commodity A or B is located by other image recognition means. In the case where it is determined that the live video contains the commodity A or the commodity B, it is determined that the live video is an illegal video. In the case where the review result is yes, the distribution of the live video on the live server can be intercepted, the live video can be reported to the security center of the live platform, an alarm information can be sent to the recorder of the live video, and the live video can be marked as a forbidden video. The above review process can also be completed by manual, and the review result is made by manual.

[0128] In an optional embodiment, on a video live platform, the classification object belonging to the default classification and / or the classification object belonging to the incremental type can be set as illegal information, which can be illegal behavior or use of prohibited articles made by the host. By detection and recognition, the illegal behavior and / or prohibited articles in the live video are obtained, and the illegal behavior and / or prohibited articles of the live video are further reviewed to obtain the review result of whether the live video indeed contains the illegal behavior and / or prohibited articles. If yes, the live video is intercepted, reported, alarmed and marked.

[0129] In step S404, the object retained in the review result is added with label information.

[0130] The object retained in the review result can be that the live video does not contain the classification object belonging to the default classification and / or the classification object belonging to the incremental type confirmed by the above review process, that is, the result of the classification detection of the live video to be recognized by the target classification model is inaccurate.

[0131] The label information can be used to correct the classification of the reserved object. For example, in a live video platform of an e-commerce, a classification object belonging to a default classification can be a product A, and a classification object belonging to an incremental type can be a product B. A target classification model detects a live video and identifies that the live video contains the product A. In the above review process, it is confirmed that the live video is actually a live video of the product C. It is determined that the live video is a reserved object, and the product C is taken as the label information of the video. In an optional embodiment, the reserved object with the added label information can be used as sample data to train the target classification model, so that the target classification model can accurately classify the product A and the product C, and improve the recognition accuracy of the classification object belonging to the default classification and / or the classification object belonging to the incremental type.

[0132] Embodiment 3

[0133] According to the embodiments of the present application, a data processing method is also provided, Figure 5 As shown in the flowchart of the data processing method according to Embodiment 3 of the present application, Figure 5 The processing method comprises the following steps:

[0134] In step S501, traffic videos of a road section in a predetermined time period are collected, wherein the traffic videos contain vehicles passing through the road section in the predetermined time period.

[0135] The traffic video is a video that needs to be detected and classified. In the application of intelligent traffic, the vehicles in the traffic video are classified and detected to identify different types of vehicles, and the identification result can be used for the management of intelligent traffic. The predetermined time period can be set according to the traffic management rules, which is not limited here.

[0136] In step S502, a target classification model is used to classify and detect the traffic video, and at least a vehicle belonging to a default type and at least one vehicle belonging to an incremental type are detected from the traffic video; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the vehicle of the incremental type, and the initial classification model is a model allowing to identify a vehicle satisfying the default type from any video.

[0137] The vehicle of the default type is a vehicle type that can be classified and detected by the above-mentioned initial classification model, and the vehicle of the incremental type is a vehicle type that cannot be identified by the initial classification model, i.e. a vehicle that can be identified after optimization and training of the initial classification model. For example, in a traffic video, the vehicle of the default type can be a small car, and the initial classification model of the traffic processing platform can detect the small car in the traffic video, identify the small car, and the vehicle of the incremental type can be a large car, i.e. a vehicle type that cannot be identified by the initial classification model.

[0138] The backbone network is a feature extraction network for feature extraction of a video image. The target classification model adopts the same backbone network as the initial classification model. Specifically, in a case where the target classification model has not been trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network for classification detection of the features extracted by the backbone network.

[0139] In the application of intelligent traffic, the above traffic video can be a traffic video of a main road in a city during rush hours, the default type of vehicle can be a small car, the initial classification model can identify the small car from the traffic video, and be used to count the traffic flow of the main road in the city during rush hours. If large trucks are not allowed to enter the main road in the city during rush hours, the large truck is determined as an incremental type of vehicle, the initial classification model is optimized and trained to obtain a target classification model, the target classification model can not only identify the small car but also identify the large truck, the target classification model is used to identify the traffic video of the main road in the city during rush hours, and identify the large truck that violates the rules to enter the main road, and further records the illegal information of the large truck according to the vehicle information (such as the license plate number) of the large truck in the traffic video, thereby realizing intelligent management of urban traffic.

[0140] Embodiment 4

[0141] According to the embodiments of the present application, a data processing method is also provided, Figure 6 As shown in the flowchart of the data processing method according to Embodiment 4 of the present application, Figure 6 The processing method comprises the following steps:

[0142] Step S601, displaying a live video to be identified in a live interface, wherein different types of products are played in the live video.

[0143] The live interface can be a live interface of an e-commerce live platform, and the different types of products are products to be sold in the live video. The live video is a video that needs to be classified and detected.

[0144] Step S602, in response to a detection instruction sensed in the live interface, calling a target classification model.

[0145] The detection instruction can be issued by a user on a live interface. For example, when a user watches live videos of multiple products on a live interface of an e-commerce live platform, the user can perform corresponding operations on the live interface to issue detection instructions for different types of products in the live videos to identify the categories of the multiple products. The detection instruction can also be automatically issued before playing the live videos. For example, the detection instruction can be automatically triggered before playing the live videos to detect the categories of the multiple products in the live videos.

[0146] At step S603, the detection result of the live video is displayed in the live interface. The live video is classified and detected by using the target classification model. The detection result includes products belonging to the default type and at least one product belonging to the incremental type in the live video. The target classification model includes at least the same backbone network as the initial classification model and a model for identifying the product of the incremental type. The initial classification model is a model that allows to identify products meeting the default type from any video.

[0147] Specifically, the product of the default type is a product category that can be classified and detected by the initial classification model. The product of the incremental type is a category that cannot be identified by the initial classification model, i.e., a newly added category that can be identified after the initial classification model is optimized and trained. For example, the product of the default type in the live interface of the e-commerce live platform can be product A that can be identified. The live video platform can identify product A in the live video collected during the live process. At this time, the live video platform cannot identify product B, which is the product of the incremental type. By training and optimizing the initial classification model already available in the live video platform, the target classification model is obtained, so that the live video platform can identify product A and product B.

[0148] The detection result can include the product of the default type, the product of the incremental type, and identification information. For example, after the target classification model detects product A in the live video, product A can be framed in the live video with an identification frame and a text identification "product A" is added to realize the display of product A on the live platform. When the product of the default type and the product of the incremental type include multiple types of products, multiple products can be identified in the live video and corresponding identifications are added to facilitate the user to quickly find the required product in the live video.

[0149] The backbone network is a feature extraction network for extracting features of a live video image. The target classification model adopts the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network for classifying and detecting the features extracted by the backbone network.

[0150] In an optional embodiment, in the application of an e-commerce live broadcast platform, the video to be recognized is a product sales live broadcast video on the live broadcast platform, the initial classification model is a model that has been put online on the live broadcast platform and can be used to detect an existing product A. When it is necessary to detect a product B on the live broadcast platform, since the current initial classification model cannot detect the product B, a target classification model with the same backbone network as the initial classification model is constructed according to the structure of the initial classification model, and the target classification model is trained using sample images with label information of the product B. The trained target classification model can replace the initial classification model to detect the product A and the product B on the live broadcast platform, realizing the expansion of the detection categories on the live broadcast platform without increasing the number of classification models, and saving the deployment cost of the classification models on the platform. In addition, the product A and the product B can be framed out on the live broadcast video and an identification is added as a detection result for display, facilitating users to quickly find the required product in the live broadcast video.

[0151] Embodiment 5

[0152] According to the embodiments of the present application, an embodiment of a data processing method is also provided, Figure 7 As shown in the flowchart of the data processing method according to Embodiment 5 of the present application, Figure 7 The processing method comprises the following steps:

[0153] In step S701, the course content to be recognized is obtained from the teaching video. The course content displays different types of teaching content.

[0154] The teaching video can be a video of each teaching platform on the Internet, and a user can watch the teaching video on the operation interface of a client. The categories of the teaching content contained in the course content to be recognized can be different disciplines (for example, mathematics, English, etc.), or different topics or chapters in the same discipline.

[0155] Step S702, in response to the detection instruction sensed in the operation interface of the teaching video, the target classification model is called.

[0156] The detection instruction can be issued by the user on the operation interface, for example, the user watches the teaching video on the operation interface of the teaching video, and issues the detection instruction for different types of teaching content in the teaching video by performing corresponding operations on the operation interface, to identify the category of the teaching content. The detection instruction can also be triggered automatically by the operation interface of the teaching video, for example, the detection instruction can be set to be triggered automatically before the teaching video is played, to detect the category of the teaching content in the teaching video.

[0157] Step S703, the detection result of the detected course content is displayed in the teaching video, wherein the target classification model is used to classify and detect the course content, and the detection result includes: the teaching content in the course content belonging to the default type and at least one teaching content belonging to the incremental type; wherein the target classification model at least includes the same backbone network as the initial classification model, and a model for identifying the teaching content of the incremental type, and the initial classification model is a model allowing to identify the teaching content meeting the default type from any video.

[0158] Specifically, the default type of teaching content is the category that can be classified and detected by the initial classification model, and the incremental type of teaching content is the category that cannot be identified by the initial classification model, i.e. the newly added category that can be identified after optimization and training of the initial classification model. For example, the default classification of teaching content can be a teaching video of a mathematics course that can be identified, and the live video platform can identify the mathematics course in the teaching video. At this time, the initial classification model cannot identify the English course, so the English course is the above-mentioned incremental type of teaching content. By training and optimizing the existing initial classification model, the target classification model is obtained, so that the live video platform can identify the mathematics course and the English course.

[0159] The detection result can include the teaching content belonging to the default type and at least one teaching content belonging to the incremental type and identification information, for example, the target classification model detects the mathematics course in the teaching video, and can add the text identification "mathematics" in the teaching video to show the theme of the teaching video. By detecting and classifying a large amount of teaching videos by the target classification model, and displaying the detection result, the user can quickly find the required course in a large number of teaching videos.

[0160] The backbone network is a feature extraction network for feature extraction of a live video image. The target classification model adopts the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network for classification detection of the features extracted by the backbone network.

[0161] In this embodiment, the target classification model obtained by optimizing the structure and parameters based on the initial classification model is used to realize the category expansion of the teaching content, without the need to redesign a new classification model, thereby realizing the dynamic expansion of the category detection of the teaching content in the teaching video platform by using only one classification model, solving the problem of increasing deployment costs caused by the need to retrain a detector when a new detection category is added in the prior art. In addition, through the display of the detection result, the user can quickly find the required teaching video.

[0162] Embodiment 6

[0163] According to the embodiments of the present application, an embodiment of a data processing method is also provided, Figure 8 The flowchart of the data processing method according to Embodiment 6 of the present application is shown in Figure 8 The processing method includes the following steps:

[0164] In step S801, a physiological video of a detection site is collected by a medical device.

[0165] The physiological video of the detection site can be an image video taken by a medical detection device such as a nuclear magnetic resonance device, or a CT (Computed Tomography), or a Doppler ultrasound device, which contains images of the detection site.

[0166] In step S802, the physiological video is displayed on the examination interface of the medical device, wherein the physiological video shows different types of pathological information.

[0167] The different types of pathological information can be pathological patterns of the detection site presented on the physiological video. For example, in the image video taken by the Doppler ultrasound device, the pathological pattern can be a shadow, and the corresponding disease can be determined according to the position of the shadow on the detection site, the size of the shadow, and the shape of the shadow.

[0168] Step S803, in response to the detection instruction sensed in the examination interface of the medical device, the target classification model is called.

[0169] The detection instruction can be issued by the doctor on the examination interface of the medical device, for example, the doctor performs corresponding operation on the examination interface of the medical device, issues the detection instruction for different types of pathological information in the physiological video, to identify the type of pathological information. The detection instruction can also be automatically issued on the examination interface of the medical device, for example, the medical device can be set to automatically trigger the detection instruction after the doctor completes the shooting of the physiological video, to detect the category of pathological information in the physiological video.

[0170] Step S804, the detection result of the physiological video in the examination interface is displayed, wherein the physiological video is detected by the target classification model, and the detection result includes the pathological information of the default type in the physiological video and at least one pathological information of the incremental type.

[0171] Specifically, the default type of pathological information is the category that can be classified and detected by the initial classification model, and the incremental type of pathological information is the category that cannot be recognized by the initial classification model, i.e. the newly added category that can be recognized after the initial classification model is optimized and trained. For example, the default classification of pathological information can be pathological pattern A, and the initial classification model can recognize pathological pattern A from the physiological video. At this time, the initial classification model cannot recognize pathological pattern B, so pathological pattern B is the above-mentioned incremental type of pathological information. By training and optimizing the existing initial classification model, the target classification model is obtained, so that the medical device can recognize pathological pattern A and pathological pattern B.

[0172] The detection result can include the default type of pathological pattern and at least one incremental type of pathological pattern and identification information, for example, the target classification model detects the pathological pattern A in the physiological video, and can frame the pathological pattern A in the physiological video with an identification box and add the text identification "pathological pattern A", to realize the identification of pathological pattern A and show the position of pathological pattern A in the physiological video to the doctor. When the default classification product and the incremental classification product contain multiple types of pathological information, multiple pathological information can be identified in the examination interface to realize the comprehensive examination of the detected part.

[0173] It should be noted that the disease diagnosis rule corresponding to the pathological pattern can be determined by the doctor in advance and stored in the medical device. After the target classification model identifies different types of pathological information, the corresponding disease can be displayed on the medical device according to the corresponding relationship between the pathological information and the disease.

[0174] Step S805, fill the detection result into the text recording the case information to obtain a structured filled case text.

[0175] Step S806, display the structured filled case text on the examination interface, wherein the target classification model at least includes the same backbone network as the initial classification model and a model for identifying the incremental type of pathological information, and the initial classification model is a model allowing identification of pathological information meeting a default type from any video.

[0176] For example, the above detection result contains picture information of pathological pattern A and text identification information "pathological pattern A", "pathological pattern A" is filled into the case text, and "pathological pattern A" is displayed on the examination interface, so that the doctor can quickly obtain the pathological information contained in the physiological video without spending time to manually view the physiological video to determine the disease corresponding to the pathological information.

[0177] The above backbone network is a feature extraction network for feature extraction of live video images, and the target classification model adopts the same backbone network as the initial classification model. Specifically, in the case where the target classification model has not been trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network for classification detection of the features extracted by the backbone network.

[0178] In this embodiment, the target classification model obtained by optimizing the structure and parameters based on the initial classification model realizes the category expansion of the pathological information, without the need to redesign a new classification model, thereby realizing the dynamic expansion of the pathological information category detection in the medical equipment using only one classification model, and solving the problem of increasing deployment cost caused by the need to retrain a detector in the prior art when a new detection category is added.

[0179] Embodiment 7

[0180] According to the embodiments of the present application, an embodiment of a data processing method is also provided, Figure 9 As shown in the flowchart of the data processing method according to Embodiment 7 of the present application, Figure 9 The processing method comprises the following steps:

[0181] Step S901, the cloud server receives a detection message from the client, wherein the detection message carries identification information representing the identified video.

[0182] The cloud server is used to carry a classification model for classifying and detecting a video. The video watched by a user on a client is classified and detected by the classification model on the cloud server.

[0183] The client can be a video software or an application on a smart device (such as a mobile phone, a smart tablet, and a computer) of the user, and the user can watch a video on the Internet through the client. The user can send a detection message through a corresponding operation on the video software or the application on the client. For example, on an e-commerce shopping platform, the detection message can be sent by clicking a live video, or the detection message can be automatically sent when the user opens the live video without the user's operation.

[0184] The identification information of the video includes, but is not limited to, the name and storage address of the video. The cloud server can determine the video that needs to be classified and detected according to the identification information of the video.

[0185] In step S902, the cloud server acquires the video to be identified based on the identification information.

[0186] After the cloud server determines the video that needs to be classified and detected according to the identification information of the video, the video to be identified can be acquired from the client, or the position of the video on the video live platform of the cloud server can be determined according to the name of the video to obtain the video to be identified.

[0187] In step S903, the cloud server classifies and detects the video to be identified by using the target classification model to obtain a detection result, wherein the detection result includes a classification object belonging to a default type and at least one classification object belonging to an incremental type in the video to be identified.

[0188] The classification object of the default classification is a category that can be classified and detected by the initial classification model, and the classification object of the incremental type is a category that cannot be recognized by the initial classification model, that is, a newly added category that can be recognized after the initial classification model is optimized and trained.

[0189] For example, in the implementation of a live video, the classification object of the default classification can be a product A that can be recognized by a live video platform. The live video platform can recognize the product A in a live video collected in a live process. At this time, the live video platform cannot recognize a product B, and the product B is the classification object of the incremental type. By training and optimizing the initial classification model already existing in the live video platform, the target classification model is obtained, so that the live video platform can recognize the products A and B.

[0190] The classification object of the default type and the classification object of the incremental type can each include multiple categories, that is, the initial classification model can be used to realize classification detection of multiple categories. The classification object of the default type and the classification object of the incremental type are different.

[0191] In step S904, the cloud server returns the detection result to the client; wherein the target classification model at least includes the same backbone network as the initial classification model, and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing to identify the classification object of the default type from any video.

[0192] The detection result can include a product belonging to the default type and at least one product belonging to the incremental type and identification information. For example, after the target classification model detects product A in the live video, the product A can be framed in the image of the live video with an identification frame and a text identification "product A". The identification frame and the text identification of the product A in the video can be returned to the client as the detection result, so that the product A with the identification frame and the identification displayed on the live video can be seen on the display interface of the client.

[0193] The backbone network is a feature extraction network for extracting features of a video image. The target classification model adopts the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network, which is used to classify and detect the features extracted by the backbone network.

[0194] In the embodiment, the cloud server receives the detection message from the client and acquires the to-be-identified video based on the identification information, and classifies and detects the to-be-identified video by using a target classification model to obtain a detection result, wherein the target classification model at least includes a same backbone network as an initial classification model and a model for identifying classification objects of an incremental type, the initial classification model is a model allowing classification objects of a default type to be identified from any video, the target classification model is obtained by optimizing the structure and parameters of the initial classification model, the expansion of the detection categories is realized, a new classification model does not need to be redesigned, and the dynamic expansion of the detection categories by using only one classification model for an image field on a live video platform of the cloud server is realized, and the problem that a detector needs to be retrained when a new detection category is added in the prior art, resulting in an increase in deployment cost, is solved.

[0195] As an optional embodiment, after the cloud server returns the detection result to the client, the method further includes: the cloud server receives correction information sent by the client, wherein the correction information is used to determine whether the identification result is accurate, and the identification result includes whether the classification objects of the default type and the classification objects of the incremental type identified are accurate; in the case that the correction information indicates that the identification result is incorrect, the cloud server receives correction information sent by the client and corrects the identification result based on the correction information; and the cloud server sends the corrected identification result to the client.

[0196] The correction information can be issued by a user on the interface of the client, specifically, the user compares the detection result displayed on each frame of the video with the actual category, determines whether the detection result matches the actual category, and further determines whether the identification result is accurate. For example, the user can input, on the client, the correct identification result or the incorrect identification result as the correction information through manual comparison and judgment.

[0197] In the case that the correction information indicates that the identification result is incorrect, the user can input the correction information on the interface of the client to display the correct identification result.

[0198] For example, in the embodiment of the e-commerce live broadcast platform, after classification detection, the goods A in the video is framed with a label box. The user can compare the detection result with the actual situation to determine whether the content framed by the label box is goods A. If the framed content is actually goods B, the user sends correction information on the client that the recognition result is wrong. Further, the identification information of the framed content of the label box can be modified to "goods B" as correction information, and goods B and its identification information are displayed in the video. Alternatively, a label box can be added to the position of goods A in the image of the video and marked as "goods A" as correction information to correct the recognition result, so as to accurately display goods A and its identification information in the video. The above correction information can also include the correction information of goods A and goods B to accurately display goods A and its identification information, and goods B and its identification information.

[0199] Embodiment 8

[0200] According to the embodiments of the present application, an embodiment of a data processing method is also provided, Figure 10 As shown in the flowchart of the data processing method according to Embodiment 8 of the present application, Figure 10 The processing method comprises the following steps:

[0201] In step S1001, video recognition parameters and a live broadcast video to be recognized are obtained.

[0202] The above-mentioned video to be recognized is a video that needs to be detected and classified. The video to be recognized can be a video of any theme or application scenario, including but not limited to a live broadcast video obtained on a live broadcast platform, a traffic video of a traffic system, a teaching video in the field of education, a medical examination video in the field of medical treatment, etc.

[0203] The video recognition parameters are categories that need to be recognized in the video to be recognized, which can be set according to the user's demand. For example, after obtaining an e-commerce live broadcast video, the video recognition parameters can be set as goods A and goods B, and then the classification detection model is used to recognize the two categories of goods A and goods B in the live broadcast video to be recognized.

[0204] In the video review scenario, the live broadcast video to be recognized can be a live broadcast video on a video live broadcast platform. The above-mentioned video recognition parameters can be categories of illegal content of the video. By setting the video recognition parameters according to the demand, the content of the live broadcast video can be reviewed before the live broadcast video is distributed to the user end, to determine whether it involves the illegal content categories contained in the video recognition parameters, and then take appropriate preprocessing measures to avoid the live broadcast video containing illegal content from being spread on the network platform.

[0205] In step S1002, the initial classification model is optimized based on the live video recognition parameter to obtain a target classification model. The initial classification model is a model that allows any video to be identified to meet the first classification. The video recognition parameter includes identification information representing the second classification object.

[0206] Specifically, the first classification object can be the default type of classification object in Embodiment 1, and the second classification object can be the incremental type of classification object in Embodiment 1. The first classification object is a category that can be classified and detected by the initial classification model, and the second classification object is a category that cannot be recognized by the initial classification model, i.e., a new category that can be recognized after the initial classification model is optimized and trained.

[0207] When the video recognition parameter is set, it can include identification information of the first classification object and identification information of the second classification object, or only identification information of the second classification object. It should be noted that the first classification object and the second classification object can each include multiple categories, i.e., the initial classification model can be used to implement classification and detection of multiple categories, the sample data can be data classified by multiple categories, and the first classification object and the second classification object are different. For example, the first classification object can be goods A and goods B in an image, and the second classification object can be goods C and goods D in an image.

[0208] Optimizing the initial classification model can construct a target classification model based on the structure and parameters of the initial classification model, and improve the structure and parameters of the initial classification model, so that the target classification model can meet the requirements of classification and detection of live video according to the video recognition parameter. For example, after obtaining a live video, the video recognition parameter can be set as goods A and goods B, and the initial classification model can only recognize goods A but cannot recognize goods B. The structure and parameters of the initial classification model are optimized to obtain a target classification model, and the target classification model can detect goods B after training.

[0209] In an optional embodiment, the optimization of the initial classification model to obtain the target classification model can be that the backbone network in the target classification model reuses the backbone network of the initial classification model, and a new head network is added after the backbone network in the target classification model. The network parameters of the reused backbone network of the initial classification model are used to train the initial classification model as a pre-training network. The newly added head network in the target classification model can be used to detect the features extracted by the backbone network of the target classification model. After the training of the target classification model is completed, the newly added head network can be used to detect the first kind of classified objects and the second kind of classified objects based on the features extracted by the backbone network of the target classification model. The network parameters of the newly added head network in the target classification model can be the same as or different from the head network of the initial classification model when initially set. The number of nodes in the newly added head network in the target classification model should be equal to the number of categories in the newly added second kind of classified objects.

[0210] In step S1003, the target classification model is trained using sample data, wherein the target classification model uses the same backbone network as the initial classification model, and the sample data includes sample images and labeling information for labeling the second kind of classified objects in the sample images.

[0211] The backbone network is a feature extraction network for extracting features of video images. The target classification model uses the same backbone network as the initial classification model. Specifically, before the target classification model is trained, the network parameters of the backbone network of the target classification model are the same as the structure and network parameters of the backbone network of the initial classification model. For example, the backbone network of the initial classification model can be a 3*3 encoder-decoder structure, and the backbone network of the target classification model is also a 3*3 encoder-decoder structure. The backbone network of the target classification model and the backbone network of the initial classification model have the same up-sampling layer and down-sampling layer. The target classification model can also add a head network at the output of the backbone network to detect the features extracted by the backbone network.

[0212] The labeling information for labeling the second kind of classified objects in the sample images can be obtained by manual labeling. After the second kind of classified objects as the newly added classification are determined, the sample images are manually labeled to obtain the above-mentioned labeling information. For example, if the detection of commodity B is required, commodity B is determined as the second kind of classified object, and the "commodity B" is labeled with a frame in the sample image.

[0213] The target classification model is trained and learned using the sample image containing the label information of the second classification object, so that the target classification model can perform classification detection on the second classification object in the image. In an optional embodiment, the sample image is input into the initial classification model, the initial classification model outputs the detection result of the first classification object in the sample image, the target classification model learns the first classification object based on the distillation knowledge of the first classification object, and then the target classification model can realize classification detection of the first classification object and the second classification object.

[0214] In step S1004, the live video is classified and detected based on the trained target classification model, and the first classification object and / or the second classification object in the live video are obtained.

[0215] It should be noted that the trained target classification model can detect the first classification object and the second classification object. In actual scenarios, there are cases where the live video to be recognized contains only the first classification object or only the second classification object. Therefore, the target classification model can detect any one or several of the first classification object and the second classification object in the live video.

[0216] In an optional embodiment, in a live video platform, the live video to be recognized is a large number of live videos in the live video platform, the initial classification model is an initial classification model that has been online in the live video platform and can be used to detect the existing category A, and the video recognition parameter is set to category B according to the user's demand to propose the need to detect category B on the live video platform. However, the current initial classification model cannot detect category B. According to the structure of the initial classification model, a target classification model with the same backbone network as the initial classification model is constructed, and the target classification model is trained using sample images with label information of category B. The target classification model after training can replace the initial classification model to detect category A and category B on the live video platform, realizing the expansion of the detection category on the live video platform without increasing the number of classification models, and saving the deployment cost of the classification model on the platform.

[0217] In this embodiment, by acquiring video recognition parameters and the live video to be recognized, the initial classification model is optimized based on the video recognition parameters to obtain a target classification model. The target classification model uses the same backbone network as the initial classification model and is trained using sample data, which includes sample images and labeling information for marking objects of the second category in the sample images. Based on the trained target classification model, the live video is classified and detected to obtain objects in the live video that satisfy the first category and / or satisfy the second category. This allows for the expansion of detection categories using a target classification model optimized based on the structure and parameters of the initial classification model, without the need to redesign a new classification model. Consequently, on the live video platform, for an image domain, only the same classification model can be used for dynamic expansion of detection categories, solving the problem in the prior art where adding a new detection category requires retraining a detector, leading to increased deployment costs.

[0218] Example 9

[0219] According to embodiments of the present invention, an apparatus for implementing the above-described data processing method is also provided. Figure 11 This is a schematic diagram of a data processing apparatus according to Embodiment 9 of this application, as shown below. Figure 11 As shown, the device 1100 includes:

[0220] The video acquisition module 1101 is used to acquire the video to be identified; the classification module 1102 is used to classify and detect the video to be identified using a target classification model, and to detect at least one classification object belonging to the default type and at least one classification object belonging to the incremental type from the video; wherein, the target classification model includes at least the same backbone network as the initial classification model, and a model for identifying classification objects of the incremental type, and the initial classification model is a model that allows the identification of classification objects that satisfy the default type from any video.

[0221] It should be noted that the video acquisition module 1101 and classification module 1102 mentioned above correspond to steps S201 to S202 in Embodiment 1. The two modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run in the computing device 10 provided in Embodiment 1.

[0222] As an optional embodiment, the apparatus further comprises a data acquisition module configured to acquire sample data and video recognition parameters; and an optimization module configured to optimize the initial classification model based on the video recognition parameters to obtain a target classification model, wherein the video recognition parameters comprise identification information representing classification objects of the incremental type; and the target classification model is trained using the sample data, wherein the sample data comprises sample images and label information for labeling classification objects belonging to the incremental type in the sample images.

[0223] As an optional embodiment, the apparatus further comprises a display module configured to display the classification objects of the default type and the incremental type, and the display module comprises any one or more of the following: a label display submodule configured to label the classification objects of the default type and the incremental type in each frame of the video using different label boxes; and a frequency display submodule configured to display the number of times the classification objects of the incremental type appear in a preset time period.

[0224] As an optional embodiment, the apparatus further comprises any one or more of the following: a sample increase module configured to increase the sample data used to train the target classification model if the number of times the classification objects of the incremental type appear in a preset time period is greater than a preset number of times; and a labeling module configured to acquire a category with an identification accuracy lower than a preset threshold value and label the category with the identification accuracy lower than the preset threshold value.

[0225] As an optional embodiment, the apparatus further comprises a correction information receiving module configured to receive correction information, wherein the correction information is used to determine whether the identification result is accurate, and the identification result comprises the classification objects of the default type and the incremental type; a correction module configured to receive correction information if the correction information indicates that the identification result is incorrect, and correct the identification result based on the correction information; and a result display module configured to display the corrected identification result.

[0226] As an optional embodiment, the target classification model comprises a second backbone network and a second head network, and the second head network comprises a second classification branch and a second regression branch.

[0227] As an optional embodiment, the device further comprises a model obtaining module configured to obtain the target classification model, the model obtaining module comprising: an initial model obtaining submodule configured to obtain an initial detection model and sample data, wherein the initial detection model comprises a first backbone network configured to perform feature extraction and a first head network configured to perform classification detection of a default type of classification object, and the sample data comprises sample images and label information of a second type of classification object in the sample images; a multiplexing submodule configured to perform multiplexing processing on network parameters of the first backbone network to generate a second backbone network; an adding submodule configured to add the second head network at an output end of the second backbone network to obtain the target classification model; and a training submodule configured to train the target classification model based on the sample data, wherein the target classification model is configured to perform detection of the default type of classification object and an incremental type of classification object.

[0228] As an optional embodiment, the first head network comprises a first classification branch configured to perform classification detection of the default type of classification object and a first regression branch configured to perform bounding box regression, and the training submodule comprises: a first input submodule configured to input the sample data to the initial detection model to obtain a first classification result output by the first classification branch and a first regression result output by the first regression branch; a second input submodule configured to input the sample data to the target classification model to obtain a second classification result output by the second classification branch and a second regression result output by the second regression branch; a loss function determining submodule configured to determine a target loss function based on the first classification result, the second classification result, the first regression result, and the second regression result; and a model optimization submodule configured to optimize the target classification model based on the target loss function.

[0229] As an optional embodiment, the loss function determining submodule comprises: a first determining submodule configured to determine a classification loss and a regression loss based on the second classification result and the second regression result; a second determining submodule configured to determine a classification distillation loss based on the first classification result and the second classification result; a regression distillation loss determining submodule configured to determine a regression distillation loss based on the first regression result and the second regression result; and a loss function constituent submodule configured to determine the classification loss, the regression loss, the classification distillation loss, and the regression distillation loss to constitute the target loss function.

[0230] As an optional embodiment, the first input submodule comprises: a screening submodule configured to screen a distillation node from output nodes of the first backbone network according to a confidence score; a first output submodule configured to output the first classification result based on feature information output by the distillation node by the first classification branch; and a second output submodule configured to output the first regression result based on feature information output by the distillation node by the first regression branch.

[0231] Embodiment 10

[0232] According to an embodiment of the present application, a device for implementing the data processing method is also provided, Figure 12 FIG. 12 is a schematic diagram of a data processing device according to an embodiment of the present application, which comprises: Figure 12

[0233] The acquisition module 1201 is configured to acquire a live video to be identified collected in a live broadcast process; the detection module 1202 is configured to perform classification detection on the live video to be identified by using a target classification model, and at least detect a classification object belonging to a default type and at least one classification object belonging to an incremental type from the live video; the auditing module 1203 is configured to audit the classification object belonging to the default type and / or the classification object belonging to the incremental type, and obtain an auditing result; and the adding module 1204 is configured to add label information to the object retained in the auditing result.

[0234] It should be noted that the acquisition module 1201, the auditing module 1203, the auditing module 1203 and the adding module 1204 correspond to steps S401 to S404 in Embodiment 2, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in Embodiments 1 and 2. It should be noted that the above modules as part of the device can run in the computing device 10 provided in Embodiment 1.

[0235] Embodiment 11

[0236] According to an embodiment of the present application, a device for implementing the data processing method is also provided, Figure 13 FIG. 13 is a schematic diagram of a data processing device according to an embodiment of the present application, which comprises: Figure 13

[0237] The acquisition module 1301 is configured to acquire a traffic video of a road section in a predetermined time period, wherein the traffic video contains vehicles passing through the road section in the predetermined time period; and the detection module 1302 is configured to perform classification detection on the traffic video by using a target classification model, and at least detect a vehicle belonging to a default type and at least one vehicle belonging to an incremental type from the traffic video; wherein the target classification model at least includes a backbone network same as an initial classification model, and a model for identifying the vehicle of the incremental type, and the initial classification model is a model allowing to identify the vehicle of the default type from any video.

[0238] ​​It should be noted that the above-mentioned collection module 1301 and detection module 1302 correspond to steps S501 to S502 in Embodiment 3, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above-mentioned embodiments 1 and 3. It should be noted that the above-mentioned modules can run in the computing device 10 provided in Embodiment 1 as part of the device.

[0239] Embodiment 12

[0240] According to the embodiments of the present application, a device for implementing the above-mentioned data processing method is further provided, Figure 14 The schematic diagram of the data processing device according to Embodiment 12 of the present application is shown in Figure 14 The device 1400 comprises:

[0241] The display module 1401 is configured to display a live video to be identified in a live interface, wherein different types of products are played in the live video; the model calling module 1402 is configured to call a target classification model in response to a detection instruction sensed in the live interface; and the result display module 1403 is configured to display a detection result of the live video in the live interface, wherein the live video is classified and detected by using the target classification model, and the detection result comprises products belonging to a default type and at least one product belonging to an incremental type in the live video; wherein the target classification model at least comprises a same backbone network as an initial classification model and a model for identifying the product of the incremental type, and the initial classification model is a model allowing to identify the product meeting the default type from any video.

[0242] It should be noted that the above-mentioned display module 1401, model calling module 1402 and result display module 1403 correspond to steps S601 to S603 in Embodiment 4, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above-mentioned embodiments 1 and 4. It should be noted that the above-mentioned modules can run in the computing device 10 provided in Embodiment 1 as part of the device.

[0243] Embodiment 13

[0244] According to the embodiments of the present application, a device for implementing the above-mentioned data processing method is further provided, Figure 15 The schematic diagram of the data processing device according to Embodiment 13 of the present application is shown in Figure 15 The device 1500 comprises:

[0245] The courseware acquisition module 1501 is used to acquire courseware content to be identified from the teaching video, wherein the courseware content displays different types of teaching content; the model retrieval module 1502 is used to retrieve the target classification model in response to the detection command sensed in the operation interface of the teaching video; the courseware content display module 1503 is used to display the detection results of the detected courseware content in the teaching video, wherein the target classification model is used to classify and detect the courseware content, and the detection results include: teaching content belonging to the default type and at least one teaching content belonging to the incremental type in the courseware content; wherein the target classification model includes at least the same backbone network as the initial classification model, and a model for identifying incremental type teaching content, and the initial classification model is a model that allows the identification of teaching content that meets the default type from any video.

[0246] It should be noted that the courseware acquisition module 1501, model retrieval module 1502, and courseware content display module 1503 mentioned above correspond to steps S701 to S703 in Embodiment 5. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiments 1 and 5. It should also be noted that the above modules, as part of the device, can run in the computing device 10 provided in Embodiment 1.

[0247] Example 14

[0248] According to embodiments of the present invention, an apparatus for implementing the above-described data processing method is also provided. Figure 16 This is a schematic diagram of a data processing apparatus according to Embodiment 14 of this application, as shown below. Figure 16 As shown, the device 1600 includes:

[0249] The video acquisition module 1601 is used to acquire physiological videos of the area to be tested through medical equipment; the video display module 1602 is used to display the physiological videos on the examination interface of the medical equipment, wherein the physiological videos display different types of pathological information; the model retrieval module 1603 is used to retrieve the target classification model in response to the detection command sensed on the examination interface of the medical equipment; the display module 1604 is used to display the detection results of the physiological videos on the examination interface, wherein the target classification model is used to classify and detect the physiological videos, and the detection results include: pathological information of the default type and at least one pathological information of the incremental type in the physiological videos; the filling module 1605 is used to fill the detection results into the text recording the case information to obtain the structured filled case text; the case display module 1606 is used to display the structured filled case text on the examination interface; wherein the target classification model includes at least the same backbone network as the initial classification model, and a model for identifying incremental type pathological information, and the initial classification model is a model that allows the identification of pathological information that meets the default type from any video.

[0250] It should be noted that the video acquisition module 1601, video display module 1602, model retrieval module 1603, display module 1604, filling module 1605, and case display module 1606 mentioned above correspond to steps S801 to S806 in Embodiment 6. The six modules and their corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiments 1 and 6. It should be noted that the above modules, as part of the device, can run in the computing device 10 provided in Embodiment 1.

[0251] Example 15

[0252] According to embodiments of the present invention, an apparatus for implementing the above-described data processing method is also provided. Figure 17 This is a schematic diagram of a data processing apparatus according to Embodiment 15 of this application, as shown below. Figure 17 As shown, the device 1700 includes:

[0253] The receiving module 1701 is used for the cloud server to receive detection messages from the client, wherein the detection messages carry identification information representing the video to be identified; the video acquisition module 1702 is used for the cloud server to acquire the video to be identified based on the identification information; the classification detection module 1703 is used for the cloud server to perform classification detection on the video to be identified using a target classification model to obtain detection results, wherein the detection results include classification objects belonging to the default type and at least one classification object belonging to the incremental type in the video to be identified; the returning module 1704 is used for the cloud server to return the detection results to the client; wherein the target classification model includes at least the same backbone network as the initial classification model, and a model for identifying incremental type classification objects, and the initial classification model is a model that allows the identification of classification objects that satisfy the default type from any video.

[0254] It should be noted that the receiving module 1701, video acquisition module 1702, classification detection module 1703, and return module 1704 mentioned above correspond to steps S901 to S904 in Embodiment 7. The four modules and their corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiments 1 and 7. It should be noted that the above modules, as part of the device, can run in the computing device 10 provided in Embodiment 1.

[0255] As an optional embodiment, the above-mentioned device further includes: a correction module, used for the cloud server to receive correction information sent by the client, wherein the correction information is used to determine whether the recognition result is accurate, and the recognition result includes whether the recognized default type classification object and incremental type classification object are accurate; a correction module, used to receive correction information sent by the client when the correction information indicates that the recognition result is incorrect, and to correct the recognition result based on the correction information; and a sending module, used to send the corrected recognition result to the client.

[0256] Example 16

[0257] According to embodiments of the present invention, an apparatus for implementing the above-described data processing method is also provided. Figure 18 This is a schematic diagram of a data processing apparatus according to Embodiment 16 of this application, as shown below. Figure 18 As shown, the device 1800 includes:

[0258] The video acquisition module 1801 is configured to acquire video recognition parameters and a live video to be recognized. The optimization module 1802 is configured to optimize an initial classification model based on the video recognition parameters to obtain a target classification model, wherein the initial classification model is a model allowing recognition of an object meeting a first type of classification from any video, and the video recognition parameters include identification information representing an object meeting a second type of classification. The training module 1803 is configured to train the target classification model using sample data, wherein the target classification model adopts the same backbone network as the initial classification model, and the sample data includes sample images and labeling information for labeling the object meeting the second type of classification in the sample images. The detection module 1804 is configured to perform classification detection on the live video based on the trained target classification model to obtain the object meeting the first type of classification and / or the object meeting the second type of classification in the live video.

[0259] It should be noted that the video acquisition module 1801, the optimization module 1802, the first training module 1803, and the detection module 1804 correspond to steps S1001 to S1004 in Embodiment 8, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in Embodiments 1 and 8. It should be noted that the above modules can run in the computing device 10 provided in Embodiment 1 as part of the device.

[0260] Embodiment 17

[0261] The embodiments of the present application also provide a storage medium. Optionally, in the present embodiment, the storage medium can be used to save the program code executed by the data processing method provided in Embodiment 1.

[0262] The storage medium can be a computer-readable storage medium, which includes a stored program, wherein the program controls the device where the computer-readable storage medium is located to execute the data processing method of any one of the above when the program is running.

[0263] Optionally, in the present embodiment, the storage medium can be located in any one of the computing devices in the computing device group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0264] Optionally, the storage medium is configured to store program code for performing the following steps: acquiring a video to be recognized; and performing classification detection on the video to be recognized using a target classification model to detect at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video, wherein the target classification model at least includes the same backbone network as an initial classification model and a model for recognizing the classification object belonging to the incremental type, and the initial classification model is a model allowing recognition of the classification object meeting the default type from any video.

[0265] Optionally, the storage medium is configured to store program code for performing the following steps: before the video to be identified is classified and detected by using the target classification model, the method further comprises: obtaining sample data and video identification parameters; based on the video identification parameters, the initial classification model is optimized to obtain the target classification model, wherein the video identification parameters include identification information of the classification object representing the incremental type; the target classification model is trained by using the sample data, wherein the sample data includes: sample images and label information for labeling the classification objects belonging to the incremental type in the sample images.

[0266] Optionally, the storage medium is configured to store program code for performing the following steps: after at least one classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, the method further comprises: displaying the identified classification objects of the default type and the incremental type, including any one or more of the following: marking the classification objects of the default type and the incremental type in each frame of the video by different marking boxes; displaying the number of times the classification objects of the incremental type appear within a preset time period.

[0267] Optionally, the storage medium is configured to store program code for performing the following steps: after at least one classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, the method further comprises any one or more of the following: if the number of times the classification objects of the incremental type appear within a preset time period is greater than a preset number of times, increasing the sample data for training the target classification model; obtaining categories with an identification accuracy lower than a preset threshold, and labeling the categories with an identification accuracy lower than the preset threshold.

[0268] Optionally, the storage medium is configured to store program code for performing the following steps: after at least one classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, the method further comprises: receiving correction information, wherein the correction information is used to determine whether the identification result is accurate, and the identification result includes the identified classification objects of the default type and the incremental type; in the case that the correction information indicates that the identification result is incorrect, receiving correction information and correcting the identification result based on the correction information; displaying the corrected identification result.

[0269] Optionally, the storage medium is configured to store program code for performing the following steps: the target classification model includes: a second backbone network and a second head network, and the second head network includes: a second classification branch and a second regression branch.

[0270] Optionally, the storage medium is configured to store program code for performing the following steps: the method further comprises: obtaining the target classification model, obtaining the target classification model comprises: obtaining the initial detection model and the sample data, wherein the initial detection model comprises: a first backbone network for feature extraction and a first head network for classification detection of the default type of classification object, and the sample data comprises: sample images and label information of the second type of classification object in the sample images; performing multiplexing processing on network parameters of the first backbone network to generate a second backbone network; adding a second head network at an output end of the second backbone network to obtain the target classification model; and training the target classification model based on the sample data, wherein the target classification model is used for detection of the default type of classification object and the incremental type of classification object.

[0271] Optionally, the storage medium is configured to store program code for performing the following steps: the first head network comprises: a first classification branch for classification detection of the default type of classification object and a first regression branch for bounding box regression, and the training of the target classification model based on the sample data so that the target classification model is used for detection of the default type of classification object and the incremental type of classification object comprises: inputting the sample data into the initial detection model to obtain first classification results output by the first classification branch and first regression results output by the first regression branch; inputting the sample data into the target classification model to obtain second classification results output by the second classification branch and second regression results output by the second regression branch; determining a target loss function based on the first classification results, the second classification results, the first regression results and the second regression results; and optimizing the target classification model based on the target loss function.

[0272] Optionally, the storage medium is configured to store program code for performing the following steps: determining the target loss function based on the first classification results, the second classification results, the first regression results and the second regression results comprises: determining a classification loss and a regression loss based on the second classification results and the second regression results; determining a classification distillation loss based on the first classification results and the second classification results; determining a regression distillation loss based on the first regression results and the second regression results; and determining that the classification loss, the regression loss, the classification distillation loss and the regression distillation loss constitute the target loss function.

[0273] Optionally, the storage medium is configured to store program code for performing the following steps: inputting the sample data into the initial detection model to obtain first classification results output by the first classification branch and first regression results output by the first regression branch comprises: selecting a distillation node from output nodes of the first backbone network according to a confidence; the first classification branch outputs the first classification results based on feature information output by the distillation node; and the first regression branch outputs the first regression results based on the feature information output by the distillation node.

[0274] Embodiment 17

[0275] According to an embodiment of the present application, an embodiment of a computer terminal is also provided, which can be any one of the computer terminal devices in the computer terminal group. Alternatively, in the present embodiment, the computer terminal can be replaced by a mobile terminal or other terminal device.

[0276] Alternatively, in the present embodiment, the computer terminal can be located in at least one of the network devices in the computer network.

[0277] In the present embodiment, the computer terminal can execute the program code of the following steps in the data processing method of the application program: obtaining a video to be identified; using a target classification model to perform classification detection on the video to be identified, and detecting at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video; wherein the target classification model at least includes a same backbone network as an initial classification model, and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing to identify the classification object satisfying the default type from any video.

[0278] Alternatively, Figure 19 is a structural block diagram of a computer terminal according to an embodiment 17 of the present application, as Figure 19 shown, the computer terminal 1900 can include one or more (only one is shown in the figure) processors 1902, a memory 1904, and a peripheral interface 1906.

[0279] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and device in the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, i.e. implements the above-mentioned data processing method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal 1900 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0280] The processor is configured to run a program, and the program can call information and application programs stored in the memory through the transmission device to perform the following steps: obtaining a video to be recognized; and performing classification detection on the video to be recognized by using a target classification model, and detecting at least a classification object belonging to a default type and at least one classification object belonging to an incremental type from the video; wherein the target classification model at least includes a same backbone network as an initial classification model and a model for identifying the classification object of the incremental type, and the initial classification model is a model allowing the classification object meeting the default type to be recognized from any video.

[0281] Those skilled in the art can understand that, Figure 19 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. Figure 19 It does not limit the structure of the electronic device. For example, the computer terminal 1900 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 19 It does not limit the structure of the electronic device. For example, the computer terminal 1900 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 19 It does not limit the structure of the electronic device. For example, the computer terminal 1900 can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0282] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device by a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0283] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0284] In the above embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0285] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0286] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0287] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0288] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0289] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized in that, include: Obtain the video to be identified; The target classification model is used to classify and detect the video to be identified, and at least one classification object belonging to the default type and at least one classification object belonging to the incremental type are detected from the video, and the number of times the incremental type classification object appears is displayed. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

2. The method according to claim 1, characterized in that, Before using a target classification model to classify and detect the video to be identified, the method further includes: Acquire sample data and video recognition parameters; Based on the video recognition parameters, the initial classification model is optimized to obtain the target classification model, wherein the video recognition parameters include identification information representing the classification object of the incremental type; The target classification model is trained using the sample data, wherein the sample data includes: sample images and labeling information for labeling classification objects in the sample images that belong to the incremental type.

3. The method according to claim 1, characterized in that, After detecting at least one category object of the default type and at least one category object of the incremental type from the video, the method further includes: displaying the identified category objects of the default type and the category objects of the incremental type, including any one or more of the following: The default type of classification object and the incremental type of classification object are marked in each frame of the video using different marker boxes; Displays the number of times the incremental type of category object appears within a preset time period.

4. The method according to claim 1, characterized in that, After detecting at least one category object of the default type and at least one category object of the incremental type from the video, the method further includes: Receive correction information, wherein the correction information is used to determine whether the recognition result is accurate, and the recognition result includes the identified classification objects of the default type and the classification objects of the incremental type; If the correction information indicates that the identification result is incorrect, correction information is received, and the identification result is corrected based on the correction information; The corrected recognition results are displayed.

5. The method according to claim 1, characterized in that, The target classification model includes a second backbone network and a second head network, wherein the second head network includes a second classification branch and a second regression branch.

6. The method according to claim 5, characterized in that, The method further includes obtaining the target classification model, wherein obtaining the target classification model includes: Acquire an initial detection model and sample data, wherein the initial detection model includes: a first backbone network for feature extraction and a first head network for classification detection of objects of default type; the sample data includes: sample images and labeling information of second category objects in the sample images. The network parameters of the first backbone network are reused to generate the second backbone network; By adding the second head network to the output of the second backbone network, a target classification model is obtained. The target classification model is trained based on the sample data, wherein the target classification model is used to detect the default type of classification objects and the incremental type of classification objects.

7. The method according to claim 6, characterized in that, The first head network includes: a first classification branch for classifying and detecting objects of the default type and a first regression branch for performing bounding box regression. The target classification model is trained based on the sample data. The target classification model includes: The sample data is input into the initial detection model to obtain the first classification result output by the first classification branch and the first regression result output by the first regression branch. The sample data is input into the target classification model to obtain the second classification result output by the second classification branch and the second regression result output by the second regression branch. The target loss function is determined based on the first classification result, the second classification result, the first regression result, and the second regression result; The target classification model is optimized based on the target loss function.

8. The method according to claim 7, characterized in that, The target loss function is determined based on the first classification result, the second classification result, the first regression result, and the second regression result, including: The classification loss and regression loss are determined based on the second classification result and the second regression result; The classification distillation loss is determined based on the first classification result and the second classification result; The regression distillation loss is determined based on the first regression result and the second regression result; The classification loss, the regression loss, the classification distillation loss, and the regression distillation loss constitute the target loss function.

9. The method according to claim 7, characterized in that, The sample data is input into the initial detection model to obtain the first classification result output by the first classification branch and the first regression result output by the first regression branch, including: Distillation nodes are selected from the output nodes of the first backbone network based on confidence level; The first classification branch outputs the first classification result based on the feature information output by the distillation node; The first regression branch outputs the first regression result based on the feature information output by the distillation node.

10. A data processing method, characterized in that, include: Acquire the live video to be identified during the live broadcast; The target classification model is used to classify and detect the live video to be identified. At least one category object belonging to the default type and at least one category object belonging to the incremental type are detected from the live video, and the number of times the incremental type category object appears is displayed. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. Review the category objects belonging to the default type and / or the category objects belonging to the incremental type, and obtain the review results; Add tag information to the objects retained in the audit results; The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

11. A data processing method, characterized in that, include: Collect traffic video of a road segment within a predetermined time period, wherein the traffic video includes vehicles passing through the road segment within the predetermined time period; The traffic video is classified and detected using a target classification model. At least one vehicle belonging to the default type and at least one vehicle belonging to the incremental type are detected from the traffic video, and the number of times the incremental type classification object appears is displayed. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

12. A data processing method, characterized in that, include: The live video to be identified is displayed on the live streaming interface, wherein different types of products are played in the live video; In response to the detection command sensed in the live streaming interface, the target classification model is retrieved; The detection results of the live video are displayed in the live streaming interface. The target classification model is used to classify and detect the live video. The detection results include: products belonging to the default type and at least one product belonging to the incremental type in the live video, and the number of times the incremental type classification object appears is displayed. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

13. A data processing method, characterized in that, include: The courseware content to be identified is obtained from the teaching video, wherein the courseware content displays different types of teaching content; In response to the detection command sensed in the operation interface of the teaching video, the target classification model is retrieved; The teaching video displays the detection results of the courseware content, wherein the target classification model is used to classify and detect the courseware content, and the detection results include: teaching content belonging to the default type and at least one teaching content belonging to the incremental type in the courseware content, and displays the number of times the incremental type classification object appears; If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

14. A data processing method, characterized in that, include: Physiological videos of the area to be tested are collected using medical equipment; The physiological video is displayed on the examination interface of the medical device, wherein the physiological video shows different types of pathological information; In response to the detection command sensed in the examination interface of the medical device, the target classification model is retrieved; The inspection interface displays the detection results of the physiological video, wherein the physiological video is classified and detected using the target classification model. The detection results include: pathological information belonging to the default type and at least one pathological information belonging to the incremental type in the physiological video, and the number of times the incremental type classification object appears is displayed. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The test results are filled into the text containing the case information to obtain the structured filled case text. The structured, filled-in case text is displayed on the inspection interface; The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

15. A data processing method, characterized in that, include: The cloud server receives a detection message from the client, wherein the detection message carries identification information that represents the identified video; The cloud server obtains the video to be identified based on the identification information; The cloud server uses a target classification model to classify and detect the video to be identified, and obtains detection results. The detection results include classification objects of the default type and at least one classification object of the incremental type in the video to be identified, and display the number of times the incremental type classification object appears. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The cloud server returns the detection result to the client; The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

16. The method according to claim 15, characterized in that, After the cloud server returns the detection result to the client, the method further includes: The cloud server receives correction information sent by the client, wherein the correction information is used to determine whether the recognition result is accurate, and the recognition result includes whether the recognized default type classification object and the incremental type classification object are accurate; If the correction information indicates that the recognition result is incorrect, the system receives correction information sent by the client and corrects the recognition result based on the correction information. The corrected recognition result is sent to the client.

17. A data processing method, characterized in that, include: Obtain video recognition parameters and the live video to be recognized; Based on the live video recognition parameters, the initial classification model is optimized to obtain the target classification model. The initial classification model is a model that allows the identification of objects that satisfy the first classification from any video. The video recognition parameters include identification information representing objects of the second classification. A target classification model is trained using sample data. This model is used to detect objects of default type and objects of incremental type. The target classification model includes the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying objects of the default type, and a second classification branch and a second regression branch for identifying objects of the incremental type. The sample data includes sample images and labeling information for labeling objects of the second type in the sample images. Based on the trained target classification model, the live video is classified and detected to obtain objects in the live video that satisfy the first type and / or satisfy the second type, and the number of times the incremental type objects appear is displayed. If the number of times the second type of classification object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled.

18. A data processing apparatus, characterized in that, include: The video acquisition module is used to acquire the video to be identified. The classification module is used to classify and detect the video to be identified using a target classification model, detect at least one classification object of the default type and at least one classification object of the incremental type from the video, and display the number of times the incremental type classification object appears. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

19. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire the live video to be identified during the live broadcast. The detection module is used to classify and detect the live video to be identified using a target classification model, detect at least one classification object of the default type and at least one classification object of the incremental type from the live video, and display the number of times the incremental type classification object appears. If the number of times the incremental type of classified object appears within a preset time period is greater than a preset number, additional sample data is added for training the target classification model; and / or, categories with recognition accuracy lower than a preset threshold are obtained and labeled. The review module is used to review category objects belonging to the default type and / or category objects belonging to the incremental type, and obtain the review results. An additional module is added to add tag information to the objects retained in the audit results; The target classification model is used to detect default type classification objects and incremental type classification objects. The target classification model includes at least the same backbone network as the initial classification model, a first classification branch and a first regression branch for identifying the default type classification objects, and a second classification branch and a second regression branch for identifying the incremental type classification objects.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the data processing method according to any one of claims 1 to 17.

21. A processor, characterized in that, The processor is used to run a program, wherein the program, when running, performs the data processing method according to any one of claims 1 to 17.

Images