Multimedia data processing method, device and equipment

Abstract

The embodiment of the invention relates to a multimedia data processing method, device and equipment. The method comprises the following steps: acquiring multimedia data generated in real time in a target course from the beginning of the target course; teaching analysis processing is carried out on the multimedia data, at least one type of course structured data is obtained, and the types of the course structured data are different; for each type of course structured data, serializing the course structured data according to a preset serialization period and a corresponding target serialization mode, obtaining a binary file and storing the binary file in a disk space; after the target course is finished, reading binary files corresponding to the at least one type of course structured data from the disk space; and deserializing the read binary files and then summarizing the binary files to obtain course summarized data corresponding to the target course. By adopting the method, the generation stability of the course summary data can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a multimedia data processing method, device and equipment. Background Art

[0002] With the development of computer technology, the educational process in schools has gradually become intelligent. By deploying acquisition equipment and data processing equipment in the classroom, the acquisition equipment collects the multimedia data generated in real time in each class, and the data processing equipment performs teaching analysis on the collected multimedia data, the course summary data of each class can be obtained.

[0003] In traditional technology, the data generated by teaching analysis of multimedia data is mainly stored in the memory space of the data processing device. However, the volume of multimedia data is usually large, so the effective utilization rate of the storage space of the data processing device will be reduced. Moreover, once the data processing device is abnormally powered off or the program is restarted, the data stored in the memory space will be lost, resulting in the inability to smoothly generate the course summary data of the course, resulting in low stability in the generation of the course summary data. Summary of the invention

[0004] Based on this, it is necessary to provide a multimedia data processing method, device and equipment that can improve the stability of course summary data generation in response to the above technical problems.

[0005] In a first aspect, the present application provides a multimedia data processing method, which is applied to a data processing device, and the method includes:

[0006] When the target course starts, the multimedia data generated in real time in the target course is acquired;

[0007] Performing teaching analysis and processing on the multimedia data to obtain at least one type of course structured data, wherein each type of course structured data is different;

[0008] For each type of course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, and the binary file is stored in a disk space;

[0009] After the target course is finished, reading binary files corresponding to at least one type of course structured data from the disk space;

[0010] Deserialize the read binary files and summarize them to obtain the course summary data corresponding to the target course.

[0011] In one embodiment, for each type of course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including:

[0012] For each type of course structured data, if the total length of the data corresponding to the course structured data meets the relatively fixed length condition, the course structured data is serialized according to the preset serialization cycle and the appended serialization method corresponding to the course structured data to obtain a binary file corresponding to the course structured data;

[0013] Among them, after the target course ends, the number of binary files corresponding to the course structured data stored in the disk space is one.

[0014] In one embodiment, for each type of course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including:

[0015] For each type of course structured data, if the total length of the data corresponding to the course structured data does not meet the relatively fixed length condition, the course structured data is serialized according to the preset serialization cycle and the repetitive serialization method corresponding to the course structured data to obtain a binary file corresponding to the course structured data;

[0016] Among them, after the target course ends, the number of binary files corresponding to the course structured data stored in the disk space is equal to the number of serialization cycles.

[0017] In one embodiment, multimedia data is subjected to teaching analysis and processing to obtain at least one course structured data, including:

[0018] Through the pre-set artificial intelligence algorithm, the multimedia data is processed for teaching analysis to obtain the artificial intelligence structured data corresponding to the multimedia data;

[0019] Perform business processing based on artificial intelligence structured data to obtain at least one course structured data.

[0020] In one embodiment, the multimedia data includes video data; the course structured data includes at least one of a face library, a body library, student behavior analysis data, and teacher behavior analysis data.

[0021] In one embodiment, the multimedia data includes audio data; and the course structured data includes at least one of audio recognition text and sound events.

[0022] In a second aspect, the present application provides a multimedia data processing device, which is applied to a data processing device, and the device includes:

[0023] An acquisition module, used to acquire multimedia data generated in real time in a target course from the start of the target course;

[0024] A processing module, used for performing teaching analysis and processing on the multimedia data to obtain at least one course structured data;

[0025] A serialization module is used to serialize the course structured data according to a preset serialization cycle and a target serialization method corresponding to the course structured data for each type of course structured data, obtain a binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data in a disk space of a data processing device;

[0026] A reading module, used for reading a binary file corresponding to at least one type of course structured data from a disk space after the target course ends;

[0027] The deserialization module is used to deserialize and aggregate the read binary files to obtain the course summary data corresponding to the target course.

[0028] In a third aspect, the present application provides a data processing device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the method embodiments of the present application are implemented.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the method embodiments of the present application.

[0030] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps in the method embodiments of the present application when the computer program is executed by a processor.

[0031] The above-mentioned multimedia data processing method, device and equipment, when the target course starts, the data processing device obtains the multimedia data generated in real time in the target course; the multimedia data is processed for teaching analysis to obtain at least one course structured data; for each type of course structured data, according to the preset serialization cycle and the target serialization method corresponding to the course structured data, the course structured data is serialized to obtain the binary file corresponding to the course structured data, and the binary file corresponding to the course structured data is stored in the disk space of the data processing device; after the target course ends, the binary file corresponding to at least one course structured data is read from the disk space; the binary file corresponding to at least one course structured data is deserialized and aggregated to obtain the course summary data corresponding to the target course.

[0032] Compared with the traditional multimedia data processing method, the present application analyzes, structures and serializes the multimedia data into binary files, and stores the binary files persistently in the disk space of the data processing device. The structuring and serialization of data effectively reduces the storage space occupied by binary files, so that the effective utilization rate of the storage space of the data processing device can be effectively improved. The binary files are stored in the disk space of the data processing device, so that the binary files will not be lost even if the data processing device is abnormally powered off or the program is restarted. If the data processing device is abnormally powered off or the program is restarted, the binary file can be directly read from the disk space of the data processing device for deserialization, and the course summary data corresponding to the target course can be generated, thereby improving the stability of the generation of course summary data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is an application environment diagram of a multimedia data processing method in an embodiment;

[0034] Figure 2 is a flowchart of a multimedia data processing method in one embodiment;

[0035] Figure 3 A schematic diagram of a binary file generated based on an append-type serialization method in one embodiment;

[0036] Figure 4 A schematic diagram of a binary file generated based on a repetitive serialization method in one embodiment;

[0037] Figure 5 is a flowchart of a multimedia data processing method in another embodiment;

[0038] Figure 6 is a structural block diagram of a multimedia data processing device in one embodiment;

[0039] Figure 7 FIG. 4 is a diagram showing the internal structure of a data processing device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] In traditional technology, the data generated by teaching analysis of multimedia data is mainly stored in the memory space of the data processing device. However, once the data processing device is abnormally powered off or the program is restarted, the data stored in the memory space will be lost and can never be recovered, resulting in the inability to smoothly generate the course summary data of the course, resulting in low stability in the generation of course summary data.

[0042] In addition, each class is relatively long, for example, a class is 40 minutes long. Teaching analysis of all multimedia data collected in a class will generate a large amount of data. The memory space of data processing equipment is generally scarce and usually cannot support such a large memory usage, which affects the data processing efficiency of the data processing equipment and even causes data processing equipment failure.

[0043] In response to the above problems, an embodiment of the present application provides a multimedia data processing method, which performs teaching analysis, structures, and serializes the multimedia data in the target course into a binary file, and stores the binary file persistently in the disk space of the data processing device. In this way, the effective utilization rate of the storage space of the data processing device can be effectively improved, and the security of the data can be guaranteed. If the data processing device is abnormally powered off or the program is restarted, the binary file can be directly read from the disk space of the data processing device for deserialization, and the course summary data corresponding to the target course can be generated, thereby improving the stability of the generation of the course summary data.

[0044] The multimedia data processing method provided in this application can be applied to Figure 1 In the application environment shown.

[0045] like Figure 1 As shown, the data processing device 102 and at least one collection device 100 ( Figure 1 , and the data processing device 102 and each of the data collection devices 100 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present application. Similarly, the data processing device 102 and the server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present application.

[0046] At least one acquisition device 100 is deployed around a room (such as a classroom, conference room, etc.) to collect image and audio data of users such as teachers, students, and participants. The acquisition device 100 can be a device that integrates image acquisition and audio acquisition, or an image acquisition device, or an audio acquisition device, which is not specifically limited in the embodiments of the present application.

[0047] Server 104 can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, cloud security, host security and other network security services, CDN, as well as basic cloud computing services such as big data and artificial intelligence platforms.

[0048] The data storage system can be set up separately and can store data that needs to be processed by the server 104. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers.

[0049] The data processing device 102 is used to obtain multimedia data generated in real time in the target course at the beginning of the target course, and perform teaching analysis and processing on the multimedia data to obtain at least one course structured data. For each course structured data, the data processing device 102 can serialize the course structured data according to the preset serialization cycle and the target serialization method corresponding to the course structured data, obtain the binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data to the disk space of the data processing device. After the target course ends, the data processing device 102 can read the binary file corresponding to each of the at least one course structured data from the disk space, and deserialize and aggregate the binary file corresponding to each of the at least one course structured data to obtain the course summary data corresponding to the target course.

[0050] Optionally, the data processing device 102 may send the course summary data to the server 104 so that the server 104 can store it.

[0051] It is understood that the data processing device 102 may also send the course summary data to the next node device for processing, which is not limited in this embodiment. Figure 1 The application scenarios are only for illustration and are not limited to this.

[0052] In one embodiment, Figure 2 As shown, a multimedia data processing method is provided, which can be applied to a data processing device, comprising the following steps:

[0053] Step 202, when the target course starts, obtain the multimedia data generated in real time in the target course.

[0054] When the target course starts, the acquisition device can acquire multimedia data generated in real time in the target course and send the acquired multimedia data to the data processing device. The data processing device can receive the multimedia data sent by the acquisition device.

[0055] In one embodiment, the acquisition device may include a camera device and a microphone device, and the multimedia data includes video data acquired by the camera device and audio data acquired by the microphone device.

[0056] In one embodiment, the acquisition device may be a camera device, and the multimedia data is video data acquired by the camera device.

[0057] In one embodiment, the acquisition device may be a microphone device, and the multimedia data is audio data acquired by the microphone device.

[0058] Step 204: Perform teaching analysis on the multimedia data to obtain at least one type of course structured data, where each type of course structured data is of a different type.

[0059] Among them, course structured data is data that meets the preset data structure.

[0060] In one embodiment, the data processing device can directly perform business processing on the multimedia data to obtain at least one type of course structured data, wherein the business processing can include at least one of classification processing, aggregation processing and deduplication processing.

[0061] Step 206, for each type of course structured data, serialize the course structured data according to a preset serialization cycle and a target serialization method corresponding to the course structured data, obtain a binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data to the disk space of the data processing device.

[0062] Serialization refers to the process of converting structured data into a binary file that meets a preset serialization format. The preset serialization format may include at least one of a json format, a protobuf format, or a raw format.

[0063] In one embodiment, the data processing device pre-records the correspondence between various course structured data and their respective corresponding serialization methods. For each type of course structured data, the data processing device can serialize the course structured data according to a preset serialization cycle (for example, every one minute) and the target serialization method corresponding to the course structured data, obtain a binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data in the disk space of the data processing device.

[0064] Step 208: After the target course is finished, read the binary file corresponding to at least one type of course structured data from the disk space.

[0065] In one embodiment, after the target course is finished, the data processing device may read binary files corresponding to at least one type of course structured data one by one from the disk space.

[0066] For example, the data processing device can perform teaching analysis on the multimedia data to obtain three types of course structured data. After the target course is finished, the data processing device can read the binary files corresponding to the three types of course structured data one by one from the disk space.

[0067] For example, the binary file corresponding to the first type of course structured data is read first, then the binary file corresponding to the second type of course structured data is read, and finally the binary file corresponding to the third type of course structured data is read. This reading method can avoid the instantaneous expansion of the memory space of the data processing device.

[0068] Step 210, deserialize and aggregate the read binary files to obtain course summary data corresponding to the target course.

[0069] In one embodiment, the data processing device may deserialize the binary files corresponding to at least one course structured data to obtain at least one course structured data corresponding to the target course. Furthermore, the data processing device may aggregate the course structured data obtained by deserialization to obtain course aggregate data corresponding to the target course.

[0070] In the above-mentioned multimedia data processing method, at the beginning of the target course, the data processing device obtains the multimedia data generated in real time in the target course; the multimedia data is processed for teaching analysis to obtain at least one course structured data; for each course structured data, according to a preset serialization cycle and according to the target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, and the binary file corresponding to the course structured data is stored in the disk space of the data processing device; after the target course ends, the binary file corresponding to at least one course structured data is read from the disk space; the binary file corresponding to at least one course structured data is deserialized and aggregated to obtain the course summary data corresponding to the target course.

[0071] Compared with the traditional multimedia data processing method, the present application analyzes, structures and serializes the multimedia data into binary files, and stores the binary files persistently in the disk space of the data processing device. The structuring and serialization of data effectively reduces the storage space occupied by binary files, so that the effective utilization rate of the storage space of the data processing device can be effectively improved. The binary files are stored in the disk space of the data processing device, so that the binary files will not be lost even if the data processing device is abnormally powered off or the program is restarted. If the data processing device is abnormally powered off or the program is restarted, the binary file can be directly read from the disk space of the data processing device for deserialization, and the course summary data corresponding to the target course can be generated, thereby improving the stability of the generation of course summary data.

[0072] In one embodiment, for each type of course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including: for each type of course structured data, if the total length of the data corresponding to the course structured data meets a relatively fixed length condition, then according to a preset serialization cycle and an additional serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data; wherein, after the target course is completed, the number of binary files corresponding to the course structured data stored in the disk space is one.

[0073] Among them, the total data length corresponding to the course structured data meets the relatively fixed length condition, which means that the total data length corresponding to the course structured data will not increase in real time as the target course progresses.

[0074] For example, if the multimedia data is video data, the total length of the course structured data "face library" obtained based on the video data will not increase in real time as the target course progresses. It can be understood that the total number of students in the classroom is fixed and the total number of students will not increase in real time as the target course progresses. However, as the target course continues to collect video data, the face data in the "face library" will become more and more accurate and of higher quality.

[0075] In one embodiment, Figure 3 As shown, if the total length of the data corresponding to the course structured data meets the relatively fixed length condition, after the target course ends, the number of binary files corresponding to the course structured data stored in the disk space is one. Among them, each serialization cycle generates a serialization data, and the binary file contains multiple serialization data generated by all serialization cycles. And the beginning of the binary file is set with version information to indicate the version number of the program.

[0076] In the above embodiment, for course structured data with a relatively fixed total data length, the data in each serialization period has a strong correlation. Serializing the course structured data using an append-type serialization method can improve the accuracy of the ultimately generated course summary data.

[0077] In one embodiment, for each type of course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including: for each type of course structured data, if the total length of the data corresponding to the course structured data does not meet the relatively fixed length condition, then according to a preset serialization cycle and a repetitive serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data; wherein, after the target course is completed, the number of binary files corresponding to the course structured data stored in the disk space is equal to the number of serialization cycles.

[0078] Among them, the total length of the data corresponding to the course structured data does not meet the relatively fixed length condition, which means that the total length of the data corresponding to the course structured data will increase in real time as the target course progresses. For example, the multimedia data is video data, and the course structured data "student behavior analysis data" obtained based on the video data, the total length of the corresponding data will increase in real time as the target course progresses. It can be understood that the behavior of students in the classroom is constantly changing, and the "student behavior analysis data" obtained in each cycle of the target course is highly independent.

[0079] In one embodiment, Figure 4As shown, if the total length of the data corresponding to the course structured data does not meet the relatively fixed length condition, after the target course ends, the number of binary files corresponding to the course structured data stored in the disk space is equal to the number of serialization cycles. Among them, each serialization cycle generates a serialized data, and each binary file contains a serialized data generated by a serialization cycle. And the beginning of each binary file is set with version information to indicate the version number of the program.

[0080] In the above embodiment, for the course structured data with relatively non-fixed total data length, the data in each serialization period is highly independent, and the course structured data is serialized in a repetitive serialization manner, which can improve the data reading efficiency.

[0081] In one embodiment, multimedia data is processed for teaching analysis to obtain at least one type of course structured data, including: processing multimedia data for teaching analysis through a preset artificial intelligence algorithm to obtain artificial intelligence structured data corresponding to the multimedia data; performing business processing based on the artificial intelligence structured data to obtain at least one type of course structured data.

[0082] In one embodiment, the data processing device can perform teaching analysis and processing on the multimedia data through a preset artificial intelligence algorithm (e.g., a target detection algorithm, etc.), and obtain artificial intelligence structured data corresponding to the multimedia data, such as face frame data and body frame data, etc. The data processing device can perform business processing based on the artificial intelligence structured data, and obtain at least one type of course structured data, such as a face library, a body library, student behavior analysis data, teacher behavior analysis data, audio recognition text and sound events, etc.

[0083] In the above embodiment, the multimedia data is first processed for teaching analysis through an artificial intelligence algorithm to obtain artificial intelligence structured data corresponding to the multimedia data, and then business processing is performed based on the artificial intelligence structured data to obtain at least one course structured data, which can improve the accuracy of the course structured data.

[0084] In one embodiment, the multimedia data includes video data; the course structured data includes at least one of a face library, a body library, student behavior analysis data, and teacher behavior analysis data.

[0085] In the above embodiment, by limiting the multimedia data to include video data, and the course structured data to include at least one of a face library, a body library, student behavior analysis data, and teacher behavior analysis data, the quality of the subsequently generated course summary data can be improved.

[0086] In one embodiment, the multimedia data includes audio data; and the course structured data includes at least one of audio recognition text and sound events.

[0087] In the above embodiment, by limiting the multimedia data to include audio data, and the course structured data to include at least one of audio recognition text and sound events, the quality of the subsequently generated course summary data can be improved.

[0088] In one embodiment, the multimedia data includes video data and audio data; the course structured data includes at least one of a face library, a body library, student behavior analysis data, teacher behavior analysis data, audio recognition text, and sound events. In this way, the quality of the subsequently generated course summary data can be further improved.

[0089] In one embodiment, Figure 5 As shown, the multimedia data includes video data and audio data. When the target course starts, the camera device set in the classroom can collect the video data generated in real time in the target course, and the microphone device set in the classroom can collect the audio data generated in real time in the target course. The data processing device can obtain the video data and audio data generated in real time in the target course. The data processing device can perform image processing and related business processing on the video data to obtain at least one course structured data corresponding to the video data. And, the data processing device can perform audio processing and related business processing on the audio data to obtain at least one course structured data corresponding to the audio data. For each course structured data, the data processing device can serialize the course structured data according to the preset serialization cycle and the target serialization method corresponding to the course structured data, obtain the binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data to the disk space of the data processing device. After the target course ends, the data processing device can read the binary file corresponding to each of the at least one course structured data from the disk space, and deserialize and aggregate the binary file corresponding to each of the at least one course structured data to obtain the course summary data corresponding to the target course.

[0090] It should be understood that, although each step in the flow chart of the above-mentioned embodiments is shown in order, these steps are not necessarily performed in order. Unless there is a clear explanation in this article, the execution of these steps does not have strict order restrictions, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned embodiments may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in order, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0091] In one embodiment, Figure 6 As shown, a multimedia data processing device 600 is provided, which is applied to a data processing device, and the device specifically includes:

[0092] The acquisition module 602 is used to acquire multimedia data generated in real time in the target course when the target course starts;

[0093] The processing module 604 is used to perform teaching analysis on the multimedia data to obtain at least one course structured data, and each course structured data has a different type;

[0094] The serialization module 606 is used to serialize the course structured data according to a preset serialization cycle and a target serialization method corresponding to the course structured data for each type of course structured data, obtain a binary file corresponding to the course structured data, and store the binary file corresponding to the course structured data in the disk space of the data processing device;

[0095] The reading module 608 is used to read the binary file corresponding to at least one course structured data from the disk space after the target course ends;

[0096] The deserialization module 610 is used to deserialize and aggregate the read binary files to obtain the course summary data corresponding to the target course.

[0097] In one embodiment, the serialization module 606 is also used to serialize the course structured data for each type of course structured data, if the total data length corresponding to the course structured data meets the relatively fixed length condition, according to a preset serialization cycle and according to an additional serialization method corresponding to the course structured data, to obtain a binary file corresponding to the course structured data; wherein, after the target course is completed, the number of binary files corresponding to the course structured data stored in the disk space is one.

[0098] In one embodiment, the serialization module 606 is also used to serialize the course structured data for each type of course structured data according to a preset serialization cycle and a repetitive serialization method corresponding to the course structured data if the total data length corresponding to the course structured data does not meet the relatively fixed length condition, so as to obtain a binary file corresponding to the course structured data; wherein, after the target course is completed, the number of binary files corresponding to the course structured data stored in the disk space is equal to the number of serialization cycles.

[0099] In one embodiment, the processing module 604 is also used to perform teaching analysis and processing on the multimedia data through a preset artificial intelligence algorithm to obtain artificial intelligence structured data corresponding to the multimedia data; and perform business processing based on the artificial intelligence structured data to obtain at least one course structured data.

[0100] In one embodiment, the multimedia data includes video data; the course structured data includes at least one of a face library, a body library, student behavior analysis data, and teacher behavior analysis data.

[0101] In one embodiment, the multimedia data includes audio data; and the course structured data includes at least one of audio recognition text and sound events.

[0102] The above-mentioned multimedia data processing device, when the target course starts, the data processing device obtains the multimedia data generated in real time in the target course; the multimedia data is processed for teaching analysis to obtain at least one course structured data; for each type of course structured data, according to a preset serialization cycle and according to the target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, and the binary file corresponding to the course structured data is stored in the disk space of the data processing device; after the target course ends, the binary file corresponding to at least one course structured data is read from the disk space; the binary file corresponding to at least one course structured data is deserialized and aggregated to obtain the course summary data corresponding to the target course.

[0103] Compared with the traditional multimedia data processing method, the present application analyzes, structures and serializes the multimedia data into binary files, and stores the binary files persistently in the disk space of the data processing device. The structuring and serialization of data effectively reduces the storage space occupied by binary files, so that the effective utilization rate of the storage space of the data processing device can be effectively improved. The binary files are stored in the disk space of the data processing device, so that the binary files will not be lost even if the data processing device is abnormally powered off or the program is restarted. If the data processing device is abnormally powered off or the program is restarted, the binary file can be directly read from the disk space of the data processing device for deserialization, and the course summary data corresponding to the target course can be generated, thereby improving the stability of the generation of course summary data.

[0104] Each module in the above-mentioned multimedia data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a data processing device in the form of hardware, or can be stored in a memory in a data processing device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0105] In one embodiment, a data processing device is provided, whose internal structure diagram can be shown as follows: Figure 7 As shown. The data processing device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the data processing device is used to provide computing and control capabilities. The memory of the data processing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the data processing device is used to exchange information between the processor and an external device. The communication interface of the data processing device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a multimedia data processing method is implemented. The display unit of the data processing device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the data processing device can be a touch layer covered on the display screen, or it can be a button, trackball or touchpad set on the data processing device housing, or it can be an external keyboard, touchpad or mouse, etc.

[0106] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the data processing device to which the scheme of the present application is applied. The specific data processing device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] In one embodiment, a data processing device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0108] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0109] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0111] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0112] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A multimedia data processing method, characterized in that: The method comprises: When a target course starts, acquiring multimedia data generated in real time during the target course; Performing teaching analysis on the multimedia data to obtain at least one type of course structured data, wherein each type of the course structured data is of a different type; For each type of the course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, and the binary file is stored in a disk space; After the target course is finished, reading the binary files corresponding to the at least one course structured data from the disk space; The read binary files are deserialized and aggregated to obtain the course summary data corresponding to the target course.

2. The method according to claim 1, characterized in that For each type of the course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including: For each type of course structured data, if the total length of the data corresponding to the course structured data meets the relatively fixed length condition, then according to the preset serialization cycle and the additional serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data; Wherein, after the target course is finished, the number of binary files corresponding to the course structured data stored in the disk space is one.

3. The method according to claim 1, characterized in that For each type of the course structured data, according to a preset serialization cycle and a target serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data, including: For each type of course structured data, if the total length of the data corresponding to the course structured data does not meet the relatively fixed length condition, then according to a preset serialization cycle and according to a repetitive serialization method corresponding to the course structured data, the course structured data is serialized to obtain a binary file corresponding to the course structured data; After the target course is finished, the number of binary files corresponding to the course structured data stored in the disk space is equal to the number of serialization cycles.

4. The method according to claim 1, characterized in that The step of performing teaching analysis on the multimedia data to obtain at least one course structured data includes: The multimedia data is processed for teaching analysis by a preset artificial intelligence algorithm to obtain artificial intelligence structured data corresponding to the multimedia data; Business processing is performed based on the artificial intelligence structured data to obtain at least one course structured data.

5. The method according to any one of claims 1 to 4, characterized in that The multimedia data includes video data; the course structured data includes at least one of a face library, a body library, student behavior analysis data and teacher behavior analysis data.

6. The method according to any one of claims 1 to 4, characterized in that The multimedia data includes audio data; the course structured data includes at least one of audio recognition text and sound events.

7. A multimedia data processing device, characterized in that: Applied to a data processing device, the device comprises: An acquisition module, used for acquiring multimedia data generated in real time in a target course starting from the start of the target course; A processing module, used for performing teaching analysis on the multimedia data to obtain at least one type of course structured data; each type of the course structured data is different; A serialization module is used to serialize the course structured data according to a preset serialization cycle and a target serialization method corresponding to the course structured data for each type of the course structured data, obtain a binary file corresponding to the course structured data, and store the binary file in a disk space; A reading module, configured to read the binary files corresponding to the at least one course structured data from the disk space after the target course ends; The deserialization module is used to deserialize and aggregate the read binary files to obtain the course summary data corresponding to the target course.

8. A data processing device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images