Image data generation method and system for multiple applications and electronic equipment

Abstract

The invention relates to the technical field of image data, and discloses an image data generation method and system for multiple applications, and electronic equipment, and is applied to an application management platform, the method comprises the steps of accessing an image processing application, responding to a pre-recording request corresponding to a pre-recording application, carrying out image stream interception according to a pre-recording duration, and obtaining a pre-recorded image stream, the pre-recording request comprises a pre-recording duration, the pre-recording application is at least a part of an image processing application, in response to a video recording request corresponding to a target application, a first image stream is determined from pre-recording image streams according to the pre-recording duration corresponding to the target application so as to generate image data according to the first image stream, and the target application is any pre-recording application; according to the method, the image data of the event can be provided for multiple applications at the same time in a low-cost mode, and the method has the advantages of being high in flexibility and easy to deploy and implement.

Description

Technical Field

[0001] The present application relates to the field of image data technology, and in particular to a method, system and electronic device for generating image data for multiple applications. Background Art

[0002] In the field of computer vision, AI algorithms are increasingly being used, especially for detecting and analyzing events. Traditional AI computer vision algorithms usually use frame-by-frame analysis to process video data to identify specific events, such as perimeter intrusions or target face recognition. In this process, once the algorithm determines that an event has occurred, in addition to the event-triggered image data captured in real time, it is often necessary to obtain video footage before and after the event in order to conduct a more detailed event review and analysis. Specifically, this includes a few seconds of video before the event and a few seconds of video during the event. These video data are crucial for accurately understanding the background, dynamic process, and results of the event.

[0003] However, some existing AI vision systems face the challenge of inefficient resource utilization when implementing this function. These systems usually rely on their own built-in caching mechanism to store video frames so that they can go back and extract the required video data after an event is triggered. On the one hand, the application of independent storage of image data increases the computing power and memory burden of the system. On the other hand, it makes the video recording generation and management process complicated and inflexible, thereby increasing the cost of image data generation from two aspects.

[0004] It should be noted that the above content only provides background technical information related to the present application and does not necessarily constitute prior art. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present application provides a method, system and electronic device for generating image data for multiple applications, which provide event image data for multiple applications at the same time in a low-cost manner.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, there is provided a method for generating image data for multiple applications, which is applied to an application management platform, and the method includes: accessing an image processing application; responding to a pre-recording request corresponding to a pre-recording application, intercepting an image stream according to a pre-recording duration to obtain a pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, and the pre-recorded application is at least a part of the image processing application; responding to a recording request corresponding to a target application, determining a first image stream from the pre-recorded image stream according to the pre-recording duration corresponding to the target application to generate image data according to the first image stream, wherein the target application is any pre-recorded application.

[0008] In one embodiment of the present application, based on the aforementioned scheme, image stream interception is performed according to the pre-recorded duration to obtain a pre-recorded image stream, including: the pre-recorded request also includes an image stream source, and the image stream sources corresponding to the pre-recorded applications are the same; if the number of the pre-recorded applications is one, the pre-recorded duration corresponding to the pre-recorded application is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length, so as to obtain the pre-recorded image stream; if the number of the pre-recorded applications is multiple, the maximum pre-recorded duration is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length, so as to obtain the pre-recorded image stream.

[0009] In one embodiment of the present application, based on the aforementioned scheme, image stream interception is performed from the image stream source according to the image stream length to obtain the pre-recorded image stream, including: image frame extraction is performed from the image stream source according to the image stream length to obtain original image frames and frame types corresponding to each of the original image frames; the extracted original image frames are stored in chronological order through a preset frame buffer queue, and the first-order original image frame in the frame buffer queue is used as the first image frame; if the frame type corresponding to the first image frame does not belong to a key frame, or the image duration corresponding to the frame buffer queue is greater than or equal to the image stream length, the first image frame is deleted.

[0010] In one embodiment of the present application, based on the aforementioned scheme, a first image stream is determined from the pre-recorded image stream according to the pre-recorded duration corresponding to the target application, including: if a recording request is obtained, determining the current image frame from the pre-recorded image stream; determining the starting image frame from the original image frame before the current image frame according to the frame type, wherein the frame type of the starting image frame belongs to a key frame; in response to the starting image frame, determining the pre-recorded image stream between the starting image frame and the current image frame as an alternative image stream; if the image duration of the alternative image stream is greater than or equal to the pre-recorded duration corresponding to the target application, determining the alternative image stream as the first image stream; if the image duration of the alternative image stream is less than the pre-recorded duration corresponding to the target application, determining a new starting image frame from the original image frame before the starting image frame according to the frame type.

[0011] In one embodiment of the present application, based on the aforementioned scheme, image data is generated according to the first image stream in the following manner: the recording request includes a delay duration and an image stream source; an image stream is intercepted from the image stream source after the current image frame according to the delay duration to obtain a second image stream; data is encapsulated according to the first image stream and the second image stream to obtain the image data.

[0012] In one embodiment of the present application, based on the aforementioned scheme, the method also includes at least one of the following: intercepting the image stream according to the pre-recorded duration, and after obtaining the pre-recorded image stream, sending the pre-recorded image stream to the pre-recorded application, so that the pre-recorded application generates the recording request according to the pre-recorded image stream; after generating image data according to the first image stream, generating completion notification information, and sending the completion notification information to the target application to notify the target application that the image data generation is complete.

[0013] In one embodiment of the present application, based on the aforementioned scheme, the method further includes: if a logout request is received from any image processing application, deleting the image processing information corresponding to the image processing application, the image processing information including a pre-recording request and a recording request; if the pre-recording duration corresponding to the image processing application is greater than or equal to the maximum pre-recording duration, updating the image stream length.

[0014] According to one aspect of an embodiment of the present application, there is provided a method for generating image data for multiple applications, which is applied to image processing applications, and the method includes: accessing an application management platform; sending a pre-recording request to the application management platform, wherein the application management platform is used to intercept the image stream according to the pre-recording duration to obtain a pre-recorded image stream, and the pre-recording request includes the pre-recording duration; sending a recording request to the application management platform, wherein the application management platform is used to determine a first image stream from the pre-recorded image stream according to the pre-recording duration corresponding to the image processing application, so as to generate image data according to the first image stream.

[0015] According to one aspect of an embodiment of the present application, there is provided an image data generation system for multiple applications, comprising: an image processing application; an application management platform configured to access the image processing application; in response to a pre-recording request corresponding to the pre-recording application, image stream interception is performed according to the pre-recording duration to obtain a pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, and the pre-recorded application is at least a part of the image processing application; in response to a recording request corresponding to a target application, a first image stream is determined from the pre-recorded image stream according to the pre-recording duration corresponding to the target application to generate image data according to the first image stream, wherein the target application is any pre-recorded application.

[0016] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements an image data generation method for multiple applications as described in any one of the above embodiments.

[0017] Beneficial effects of the present application: The present application provides a method, system and electronic device for generating image data for multiple applications, which are applied to an application management platform. The method includes accessing an image processing application, responding to a pre-recording request corresponding to a pre-recording application, intercepting an image stream according to a pre-recording duration, and obtaining a pre-recorded image stream, wherein the pre-recording request includes a pre-recording duration, and the pre-recorded application is at least a part of the image processing application, responding to a recording request corresponding to a target application, determining a first image stream from the pre-recorded image stream according to the pre-recording duration corresponding to the target application, so as to generate image data according to the first image stream, wherein the target application is any pre-recorded application; after intercepting the target image stream through the application management platform, intercepting the target image frame from the target image stream can provide image data of events for multiple applications at the same time, simplifying the generation and management process of image data, improving the flexibility and availability of image data generation, and thereby reducing the management cost of image data generation.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 is a schematic diagram of a system architecture of a method for generating image data for multiple applications applied to an application management platform, shown in an exemplary embodiment of the present application;

[0021] Figure 2 is a flowchart of a method for generating image data for multiple applications applied to an application management platform, shown in an exemplary embodiment of the present application;

[0022] Figure 3 is a block diagram of an application management platform shown in an exemplary embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a registration and pre-recording process of an image data generation method for multiple applications shown in an exemplary embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a video recording process of a method for generating image data for multiple applications, shown in an exemplary embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a logout process of a method for generating image data for multiple applications according to an exemplary embodiment of the present application;

[0026] Figure 7 is a flowchart of a method for generating image data for multiple applications applied to an image processing application, shown in an exemplary embodiment of the present application;

[0027] Figure 8 is a block diagram of an image data generation system for multiple applications shown in an exemplary embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0030] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0031] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0032] First of all, it should be noted that PTS (Presentation Time Stamp) is the display timestamp, which is used to guide the player when to display the video frame. The PTS value is usually encoded together with the frame in the video stream, and the player arranges the display order and time of the frame according to this timestamp.

[0033] MP4 (MPEG-4 Part 14, MP4 format) is a multimedia computer file format using MPEG-4, with the extension .mp4, mainly used to store digital audio and digital video. The MP4 file format is a standard digital multimedia container format derived from Part 14 of the MPEG-4 standard. It contains not only video and audio data, but also subtitles and other metadata. MP4 is a digital multimedia container format used to store video, audio, subtitles and other data, and is commonly used for streaming media, video playback and storage.

[0034] Figure 1 It is a schematic diagram of the system architecture of a method for generating image data for multiple applications applied to an application management platform, shown as an exemplary embodiment of the present application.

[0035] Reference Figure 1As shown, the system architecture may include a data acquisition device 101 and a computer device 102. The computer device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The data acquisition device 101 is used to collect the connection information between the application management platform and the image processing application, as well as the pre-recording request and the recording request sent by the image processing application. In this embodiment, the data acquisition device 101 obtains the above data and provides it to the computer device 102 for processing. Relevant technicians can use the computer device 102 to control the application management platform to access the image processing application according to the connection information, respond to the pre-recording request corresponding to the pre-recording application, intercept the image stream according to the pre-recording duration, and obtain the pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, the pre-recording application is at least a part of the image processing application, respond to the recording request corresponding to the target application, determine the first image stream from the pre-recorded image stream according to the pre-recording duration corresponding to the target application, and generate image data according to the first image stream, wherein the target application is any pre-recorded application. It should be noted that the data acquisition device 101 and the computer device 102 provided in this embodiment are merely examples and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0036] It should be noted that the image data generation method for multiple applications provided in the embodiment of the present application is generally executed by the computer device 102 , and accordingly, the image data generation system for multiple applications is generally set in the computer device 102 .

[0037] Figure 2 is a flowchart of a method for generating image data for multiple applications applied to an application management platform, which is shown in an exemplary embodiment of the present application. The method for generating image data for multiple applications can be executed by a computing and processing device, which can be Figure 1 The computer device 102 shown in FIG. Figure 2 As shown, the method for generating image data for multiple applications is applied to an application management platform. The method at least includes steps S210 to S230, which are described in detail as follows:

[0038] In step S210, an image processing application is accessed.

[0039] In one embodiment of the present application, connection information between the application management platform and the image processing application is obtained, and the application management platform is controlled to access the image processing application based on the connection information. The number of the image processing application may be one or more.

[0040] In one embodiment of the present application, the process before accessing the application also includes the following steps: receiving registration requests sent by each image processing application, registering each image processing application based on the registration request corresponding to each image processing application, and sending a video source list to each image processing application so that each image processing application determines the image stream source according to the video source list.

[0041] In this embodiment, taking the image processing application as an AI (Artificial Intelligence) application as an example, each AI application initiates a registration request to the application management platform, and the application management platform generates a unique identifier AppID in response to the registration request to characterize the corresponding AI application according to each AppID; the application management platform sends a video source list to each AI application, so that each AI application determines the image stream source according to the video source list, and receives the image stream source sent by the AI ​​application.

[0042] In step S220, in response to the pre-recording request corresponding to the pre-recording application, the image stream is intercepted according to the pre-recording duration to obtain the pre-recorded image stream.

[0043] The pre-recording request includes a pre-recording duration, and the pre-recording application is at least a part of the image processing application.

[0044] In one embodiment of the present application, the process of intercepting the image stream according to the pre-recorded time to obtain the pre-recorded image stream includes the following steps: the pre-recorded request also includes an image stream source, and the image stream sources corresponding to the pre-recorded applications are the same; if the number of pre-recorded applications is one, the pre-recorded time corresponding to the pre-recorded application is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length, so as to obtain the pre-recorded image stream; if the number of pre-recorded applications is multiple, the maximum pre-recorded time is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length, so as to obtain the pre-recorded image stream.

[0045] In this embodiment, if there are multiple pre-recorded applications, the pre-recording times corresponding to each pre-recorded application are compared, and the maximum pre-recording time is determined based on the comparison result; the pre-recording times corresponding to each pre-recorded application can also be sorted in descending order, and the pre-recording time ranked first can be determined as the maximum pre-recording time.

[0046] In one embodiment of the present application, an image stream is intercepted from an image stream source according to the image stream length to obtain a pre-recorded image stream, including the following steps: image frames are extracted from the image stream source according to the image stream length to obtain original image frames and frame types corresponding to each original image frame; the extracted original image frames are stored in chronological order through a preset frame buffer queue, and the first-order original image frame in the frame buffer queue is used as the first image frame; if the frame type corresponding to the first image frame does not belong to a key frame, or the image duration corresponding to the frame buffer queue is greater than or equal to the image stream length, the first image frame is deleted.

[0047] In another embodiment of the present application, the pre-recording start time is obtained; the image stream source is fetched based on the pre-recording start time to obtain a first current frame, and the first current frame is stored in a frame cache queue; if the difference between the timestamp of the first frame in the frame cache queue and the timestamp of the first current frame is greater than the maximum pre-recording duration, or the first frame is a non-key frame, the first frame is deleted to update the frame cache queue.

[0048] In this embodiment, the pre-recording start time is determined by at least one of the following methods: determining the time when the pre-recording request is received as the pre-recording start time; obtaining the pre-recording start time from the pre-recording request, that is, the pre-recording request includes the pre-recording start time. Specifically, if it is determined that the pre-recording request does not include the pre-recording start time, the time when the pre-recording request is received is determined as the pre-recording start time. If there are multiple pre-recording applications that perform image stream interception on the same video source, that is, the image stream source, the time when the first pre-recording request is received is determined as the pre-recording start time; or, the first pre-recording start time received in the pre-recording request is determined as the pre-recording start time.

[0049] In step S230, in response to a recording request corresponding to the target application, a first image stream is determined from the pre-recorded image streams according to the pre-recording duration corresponding to the target application, so as to generate image data according to the first image stream.

[0050] The target application is any pre-recorded application.

[0051] In one embodiment of the present application, the process of determining the first image stream from the pre-recorded image stream according to the pre-recorded duration corresponding to the target application includes the following steps: if a recording request is obtained, determining the current image frame from the pre-recorded image stream; determining the starting image frame from the original image frame before the current image frame according to the frame type, wherein the frame type of the starting image frame belongs to the key frame; in response to the starting image frame, determining the pre-recorded image stream between the starting image frame and the current image frame as an alternative image stream; if the image duration of the alternative image stream is greater than or equal to the pre-recorded duration corresponding to the target application, determining the alternative image stream as the first image stream; if the image duration of the alternative image stream is less than the pre-recorded duration corresponding to the target application, determining a new starting image frame from the original image frame before the starting image frame according to the frame type.

[0052] In this embodiment, if the image duration of the candidate image stream is less than the pre-recorded duration corresponding to the target application and there is no original image frame before the start image frame, a failure message is generated and fed back to prompt the target application according to the failure message.

[0053] In this embodiment, the recording start time is obtained, and the current image frame is determined from the pre-recorded image stream based on the recording start time; the original image frames in the frame cache queue are traversed from the end to the head, and the second current frame is extracted until the difference between the timestamp of the current image frame and the timestamp of the second current frame is greater than the pre-recorded duration corresponding to the target application, and the frame type of the second current frame is a key frame, then the second current frame is determined as the starting image frame, and the pre-recorded image stream between the starting image frame and the current image frame is determined as the first image stream.

[0054] In this embodiment, the recording start time is determined by at least one of the following methods: determining the time when the application management platform receives the recording request as the recording start time; obtaining the recording start time from the recording request, that is, the recording request includes the recording start time. Specifically, if it is determined that the recording request does not include the recording start time, the time when the application management platform receives the recording request is determined as the recording start time.

[0055] In some embodiments, the determination of whether the frame type of a video frame or an image frame is a key frame by a flag is usually performed in the context of video compression coding. In video compression coding, a key frame contains enough information to independently decode and reconstruct the entire image, while a non-key frame depends on the previous frame for decoding. The following are general steps for determining whether a video frame is a key frame by a flag: First, the video stream needs to be parsed to obtain the encoding information of each video frame. This usually involves reading the binary data of the video file and parsing it according to a specific encoding format (such as H.264, H.265, etc.). In the process of parsing the video stream, it is necessary to find flags related to key frames. These flags are usually located in the header or a specific position of the video frame and are used to indicate whether the frame is a key frame. Based on the flag found, it can be determined whether the current video frame is a key frame. Different encoding formats may have different flags and judgment methods. For example, in H.264 encoding, a key frame usually has a specific start code (such as 0x65 or 0x25), or it can be determined whether the subsequent frame is a key frame by parsing the SPS (sequence parameter set) and PPS (picture parameter set). In order to verify the accuracy of the judgment result, the parsed key frames can be compared with the key frames provided by the AI ​​application. If they are consistent, it means that the judgment method is correct; if they are inconsistent, it is necessary to check the parsing process or adjust the judgment method. It should be noted that different video encoding formats and different decoder implementations may have different flags and judgment methods. Therefore, in practical applications, it is necessary to determine the judgment method based on the specific encoding format and decoder implementation. In addition, for video files that have been encoded, professional video analysis tools or software can also be used to view their encoding information and identify which frames are key frames. It can be understood that this embodiment is only for illustration and does not limit the method of identifying key frames.

[0056] In one embodiment of the present application, image data is generated based on the first image stream in the following manner: a recording request includes a delay duration and an image stream source; an image stream is intercepted from the image stream source after the current image frame according to the delay duration to obtain a second image stream; data is encapsulated based on the first image stream and the second image stream to obtain image data.

[0057] In some embodiments, based on the recording start time and the recording duration, i.e., the delay duration, an image stream is intercepted from the image stream source to obtain a second image stream; the first image stream and the second image stream are encapsulated into a preset format and stored in a preset storage space to generate image data. The preset format may be an MP4 format. Alternatively, only the first image stream may be encapsulated into a preset format and stored in a preset storage space as required, and image data may be generated based on the first image stream.

[0058] In this embodiment, the first image stream and the second image stream are packaged into a preset format and stored in a preset storage space. The order may be a time order, that is, the first image stream and the second image stream are packaged into a preset format according to the time order and stored in the preset storage space. Alternatively, the original image frames in the first image stream may be packaged into a preset format in order and stored in a preset storage space, and then the second image stream is extracted from the image stream source, and the image frames in the second image stream are packaged into a preset format and stored in a preset storage space, until the duration of the image frames extracted from the second image stream meets the recording duration. For example, if the recording duration is set to 10 seconds, the duration of the image frames extracted from the second image stream is 10 seconds, and the extraction is stopped, that is, the total duration from the start of image stream capture from the image stream source to the end of image stream capture is 10 seconds.

[0059] In one embodiment of the present application, the present application also includes at least one of the following: intercepting the image stream according to the pre-recorded time, and after obtaining the pre-recorded image stream, sending the pre-recorded image stream to the pre-recorded application, so that the pre-recorded application generates a recording request based on the pre-recorded image stream; after generating image data based on the first image stream, generating completion notification information, and sending the completion notification information to the target application to notify the target application that the image data generation is complete.

[0060] In one embodiment of the present application, if a logout request is received from any image processing application, the image processing information corresponding to the image processing application is deleted, and the image processing information includes a pre-recording request and a recording request; if the pre-recording duration corresponding to the image processing application is greater than or equal to the maximum pre-recording duration, the image stream length is updated.

[0061] In this embodiment, the image processing application sends a deregistration request to the application management platform. After receiving the deregistration request, the application management platform deletes the local registration and binding information of the application, obtains the maximum pre-recording time of the currently bound video source, i.e., the image stream source, compares the recording time of the image processing application with the maximum recording time, and updates the image stream length if the pre-recording time of the image processing application is greater than or equal to the maximum pre-recording time, so as to save memory and reduce management costs.

[0062] Figure 3 is a block diagram of an application management platform shown in an exemplary embodiment of the present application, with reference to Figure 3As shown, in an exemplary embodiment, taking the image processing application as an AI application as an example, the application management platform includes an application management module and a video recording module. Among them, the application management module provides multi-application management, video source management, and recording functions; the video recording module provides pre-recording management, video data generation and storage functions. Specifically, the video recording module takes the stream from the streaming media, and caches and updates the video frames according to the maximum value of the pre-recording time issued by each AI application and the received video frame PTS (timestamp) and frame type; after receiving the event recording request from the AI ​​application, a certain amount of video frame data is generated according to PTS, application recording time and frame type, and encapsulated as an MP4 file and stored in a preset storage space, such as a distributed object storage system such as minio object storage; the recording information is notified to the AI ​​application through a callback request.

[0063] In this embodiment, the application management module provides multi-application registration, deregistration and video source management, as well as pre-recording and recording interface functions. Specifically, after the AI ​​application registers with the application management platform, the application management platform will return the AppID to the application, and subsequent requests will use the AppID as the unique identifier; after the AI ​​application is successfully registered, it will obtain a list of all current video sources from the management module, and enable video analysis and request pre-recording for certain video sources. The application sends a pre-recording request to the application management platform, and specifies the pre-recording duration, the corresponding video source ID, and the callback notification address for the end of recording in the pre-recording request. The management module returns the real-time stream address of the video source to the application end. After receiving the pre-recording request from the AI ​​video application, the application management module sends the stream address of the image stream source and the maximum pre-recording duration of the application bound to the video source to the video recording module, and the video recording module begins to pre-record cache the video stream. After receiving the pre-recording request, the video stream address is fetched, the latest video frame data is inserted into the tail of the cache queue, i.e., the frame cache queue, and the queue is updated. The update rule of the cache queue is: after each video frame is received from the video stream address, the PTS and frame type of the head frame of the queue are determined. If the head frame PTS-current frame PTS>pre-recording duration or the head frame is a non-key frame, the head frame data is deleted. After receiving the recording request from the AI ​​application, the video frames in the video stream cache queue are processed according to the recording duration parameter. The processing rule is: traverse the cache queue from the tail to the head to fetch video frames until the tail video frame PTS-current video frame PTS>pre-recording duration sent by the application and the current video frame type is a key frame. The video frames that meet the conditions will be encapsulated into the mp4 container in sequence; the newly received video frames from the new video stream address corresponding to the image stream source are also encapsulated into the aforementioned mp4 container until the recording duration meets the requirements in the application request. For the MP4 file that has been encapsulated, the video recording module saves the file to Minio and notifies the AI ​​application of the callback. After receiving the cancellation request from the AI ​​video application, the application management module deletes the local registration information of the application, obtains the maximum pre-recording time of all current video sources bound to the application, and sends it to the video recording module. After receiving the request, the video recording module updates the pre-recording time.

[0064] In one embodiment of the present application, Figure 4 is a schematic diagram of a registration and pre-recording process of an image data generation method for multiple applications shown in an exemplary embodiment of the present application, Figure 5 is a schematic diagram of a video recording process of a method for generating image data for multiple applications, shown in an exemplary embodiment of the present application. Figure 6 is a schematic diagram of a cancellation process of a method for generating image data for multiple applications shown in an exemplary embodiment of the present application, with reference to Figures 4 to 6, taking the application as an AI application as an example, the following steps illustrate the image data generation method for multiple applications: the AI ​​application initiates registration with the application management module, the application management module generates an AppID unique identifier, and returns the AppID to the application; the AI ​​application obtains a video source list from the application management module. The AI ​​application initiates a pre-recording request to the application management module, which contains the pre-recording duration parameter, the corresponding video source ID, and the callback notification address of the end of recording. The application management module returns the corresponding video source stream address to the application end; the application management module obtains the maximum pre-recording duration of the current video source, and sends the pre-recording duration and video stream address to the video recording module; the video recording module obtains the stream according to the video stream address and caches the received video frames; wherein, the video frame cache queue update rule is: after each video frame is received from the video stream address, the PTS and frame type of the first frame of the team are judged. If the PTS of the first frame of the team-the PTS of the current frame>the pre-recording duration or the first frame of the team is a non-key frame, the frame data of the first frame of the team is deleted. The AI ​​application sends a recording request, which includes the video stream ID and recording duration. After receiving the recording request, the video recording module takes out a certain number of video frames from the corresponding cache queue according to the frame-taking rule, and puts the video frames into the mp4 container. The frame-taking rule is: traverse the cache queue from the end to the beginning to take video frames until the PTS of the video frame at the end of the queue - the PTS of the current video frame > the pre-recording duration issued by the current application and the current video frame type is a key frame. The video recording module continues to take the stream and put the video frames into the above mp4 container until the stream-taking duration meets the recording duration of the application. After the recording duration is met, the video recording module ends the stream-taking, saves the mp4 file generated by the above video frame to Minio, and sends a request to the application callback address to notify the application of the recording completion information. The application logs out from the application management module. After receiving the logout request from the AI ​​video application, the application management module deletes the local registration and binding information of the application, obtains the maximum pre-recording duration of the currently bound video source and sends it to the video recording module. After receiving the request, the video recording module updates the pre-recording duration. This application puts event video generation and management on the central side for unified management, which can solve the problem of insufficient flexibility in video generation and management of the current AI analysis system, reduce the cost of event video generation and management, and improve usability.

[0065] Figure 7 is a flowchart of a method for generating image data for multiple applications applied to an image processing application, shown in an exemplary embodiment of the present application, with reference to Figure 7 As shown, in one embodiment of the present application, the present application also provides a method for generating image data for multiple applications, which is applied to image processing applications and at least includes steps S710 to S730, which are described in detail as follows:

[0066] In step S710, access the application management platform;

[0067] In step S720, a pre-recording request is sent to the application management platform.

[0068] The application management platform is used to intercept the image stream according to the pre-recording duration to obtain the pre-recorded image stream, and the pre-recording request includes the pre-recording duration.

[0069] In step S730, a video recording request is sent to the application management platform.

[0070] The application management platform is used to determine the first image stream from the pre-recorded image streams according to the pre-recorded duration corresponding to the image processing application, so as to generate image data according to the first image stream.

[0071] It should be noted that the image data generation method for multiple applications applied to image processing applications provided in the above embodiment and the image data generation method for multiple applications applied to application management platforms provided in the above embodiment belong to the same concept, and the specific operation method of each step has been described in detail in the above embodiment and will not be repeated here.

[0072] This application can improve the generation efficiency and management flexibility of video recording data, while reducing the resource consumption and usage cost of the AI ​​video analysis system, improving the accuracy and real-time performance of event detection, and further optimizing resource utilization, thereby promoting the widespread application and development of AI technology in the field of video surveillance.

[0073] Figure 8 is a block diagram of an image data generation system for multiple applications shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown in the figure is specifically configured in the computer device 102. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.

[0074] like Figure 8 As shown, the exemplary image data generation system for multiple applications includes: an application management platform 810 and an image processing application 820 .

[0075] Among them, an image processing application 820; an application management platform 810, configured to access the image processing application 820; in response to a pre-recording request corresponding to the pre-recording application, image stream interception is performed according to the pre-recording duration to obtain a pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, and the pre-recorded application is at least a part of the image processing application 820; in response to a recording request corresponding to a target application, a first image stream is determined from the pre-recorded image stream according to the pre-recording duration corresponding to the target application to generate image data based on the first image stream, wherein the target application is any pre-recorded application.

[0076] It should be noted that the image data generation system for multiple applications provided in the above embodiment and the image data generation method for multiple applications provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In actual applications, the image data generation system for multiple applications provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0077] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the image data generation method for multiple applications provided in the above-mentioned embodiments.

[0078] Fig. 9 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0079] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 to the random access memory (RAM) 903, such as executing the methods provided in the above-mentioned various embodiments. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, the ROM 902 and the RAM 903 are connected to each other through the bus 904. The input / output (I / O) interface 905 is also connected to the bus 904.

[0080] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0081] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are executed.

[0082] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0083] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not constitute limitations on the units themselves in some cases.

[0085] Another aspect of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the image data generation method for multiple applications provided in the above-mentioned embodiments. The computer-readable storage medium may be included in the electronic device described in the above-mentioned embodiments, or may exist independently without being assembled into the electronic device.

[0086] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0087] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the image data generation method for multiple applications provided in the above-mentioned embodiments.

[0088] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0089] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0090] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for generating image data for multiple applications, characterized in that: Applied to an application management platform, the method includes: Access to image processing applications; In response to a pre-recording request corresponding to a pre-recording application, intercepting an image stream according to a pre-recording duration to obtain a pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, and the pre-recording application is at least a part of the image processing application; In response to a recording request corresponding to a target application, a first image stream is determined from the pre-recorded image streams according to a pre-recording duration corresponding to the target application, so as to generate image data according to the first image stream, wherein the target application is any pre-recorded application.

2. The method for generating image data for multiple applications according to claim 1, characterized in that: The image stream is intercepted according to the pre-recorded time to obtain the pre-recorded image stream, including: The pre-recording request also includes an image stream source, and the image stream sources corresponding to the pre-recording applications are all the same; If the number of the pre-recorded applications is one, the pre-recorded duration corresponding to the pre-recorded application is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length to obtain the pre-recorded image stream; If there are multiple pre-recorded applications, the longest pre-recorded duration is used as the image stream length, so as to intercept the image stream from the image stream source according to the image stream length to obtain the pre-recorded image stream.

3. The method for generating image data for multiple applications according to claim 2, characterized in that: The method of intercepting an image stream from the image stream source according to the length of the image stream to obtain the pre-recorded image stream comprises: Extracting image frames from the image stream source according to the image stream length to obtain original image frames and frame types corresponding to the original image frames; The extracted original image frames are stored in a preset frame buffer queue in time sequence, and the first-order original image frame in the frame buffer queue is used as the first image frame; If the frame type corresponding to the first image frame does not belong to a key frame, or the image duration corresponding to the frame buffer queue is greater than or equal to the image stream length, the first image frame is deleted.

4. The method for generating image data for multiple applications according to claim 1, characterized in that: Determining a first image stream from the pre-recorded image streams according to the pre-recorded duration corresponding to the target application includes: If a video recording request is obtained, determining a current image frame from the pre-recorded image stream; Determining a starting image frame from original image frames before the current image frame according to a frame type, wherein the frame type of the starting image frame belongs to a key frame; In response to a start image frame, determining a pre-recorded image stream between the start image frame and the current image frame as a candidate image stream; If the image duration of the candidate image stream is greater than or equal to the pre-recorded duration corresponding to the target application, determining the candidate image stream as the first image stream; If the image duration of the candidate image stream is less than the pre-recorded duration corresponding to the target application, a new starting image frame is determined from the original image frames before the starting image frame according to the frame type.

5. The method for generating image data for multiple applications according to claim 4, characterized in that: Generate image data according to the first image stream in the following manner: The video recording request includes a delay duration and an image stream source; According to the delay time, an image stream is intercepted from an image stream source after the current image frame to obtain a second image stream; Data encapsulation is performed according to the first image stream and the second image stream to obtain the image data.

6. The method for generating image data for multiple applications according to any one of claims 1 to 5, characterized in that: The method further comprises at least one of the following: Intercepting the image stream according to the pre-recording duration, and after obtaining the pre-recorded image stream, sending the pre-recorded image stream to the pre-recording application, so that the pre-recording application generates the recording request according to the pre-recorded image stream; After the image data is generated according to the first image stream, completion notification information is generated, and the completion notification information is sent to the target application to notify the target application that the image data generation is completed.

7. The method for generating image data for multiple applications according to any one of claims 1 to 5, characterized in that: The method further comprises: If a deregistration request is received from any image processing application, the image processing information corresponding to the image processing application is deleted, the image processing information including the pre-recording request and the video recording request; If the pre-recording duration corresponding to the image processing application is greater than or equal to the maximum pre-recording duration, the image stream length is updated.

8. A method for generating image data for multiple applications, characterized in that: Applied to image processing applications, the method comprises: Access to the application management platform; Sending a pre-recording request to the application management platform, wherein the application management platform is used to intercept the image stream according to the pre-recording duration to obtain the pre-recorded image stream, and the pre-recording request includes the pre-recording duration; A recording request is sent to the application management platform, wherein the application management platform is used to determine a first image stream from the pre-recorded image streams according to the pre-recording duration corresponding to the image processing application, so as to generate image data according to the first image stream.

9. An image data generation system for multiple applications, characterized in that: include: Image processing applications; an application management platform configured to access the image processing application; In response to a pre-recording request corresponding to a pre-recording application, an image stream is intercepted according to a pre-recording duration to obtain a pre-recorded image stream, wherein the pre-recording request includes the pre-recording duration, and the pre-recorded application is at least a part of the image processing application; in response to a recording request corresponding to a target application, a first image stream is determined from the pre-recorded image stream according to the pre-recording duration corresponding to the target application to generate image data based on the first image stream, wherein the target application is any pre-recorded application.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the image data generation method for multiple applications as described in any one of claims 1 to 8.

Images