Video data storage method and device, equipment, storage medium and program product

Abstract

The invention discloses a video data storage method and device, equipment, a storage medium and a program product, and the method comprises the steps: receiving video data and multi-modal semantic information sent by a camera, and the multi-modal semantic information is used for representing image features of different dimensions in the video data; according to target scene semantic information in the multi-modal semantic information, determining a target importance score corresponding to the target scene semantic information according to a preset corresponding relationship between the scene semantic information and the importance score; under the condition that the target importance score is smaller than a preset score threshold value, performing optical flow analysis on the video data to obtain an optical flow analysis result; under the condition that the video data is determined to be video frame data with the same picture content according to the optical flow analysis result, reducing the storage frame rate of the video data to be a preset first frame rate; and storing the video data according to a preset first frame rate. Compressible video data can be effectively identified, and the storage space of the video data is reduced.

Description

Technical Field

[0001] This application belongs to the field of data storage technology, and in particular relates to a method, apparatus, device, storage medium and program product for video data storage. Background Technology

[0002] With the rapid development of information technology, surveillance cameras have become an indispensable infrastructure in fields such as smart cities, home security, and industrial management. In security and safety scenarios, the widespread application of high-definition cameras and smart devices has led to an exponential increase in the demand for the collection and storage of massive amounts of video data, posing a challenge to traditional video data storage systems.

[0003] Currently, when compressing video data, the common practice is to determine whether there are significant differences in the content between adjacent video frames. Frames with identical content are then removed before compression, thus achieving compressed storage of the video data. However, the judgment logic of the above method is relatively simple and cannot effectively identify the video data that needs to be compressed, resulting in the compressed video data still occupying a large amount of storage space. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program product for storing video data, which can effectively identify compressible video data and reduce the storage space of video data.

[0005] In a first aspect, embodiments of this application provide a method for storing video data, applied to a cloud server, the method comprising: The system receives video data and multimodal semantic information transmitted from a camera, wherein the multimodal semantic information is used to represent image features of different dimensions in the video data. Based on the target scene semantic information in the multimodal semantic information, and according to the preset correspondence between scene semantic information and importance score, the target importance score corresponding to the target scene semantic information is determined; If the target importance score is less than a preset score threshold, optical flow analysis is performed on the video data to be stored to obtain the optical flow analysis results. If the video data is determined to be video frame data with the same picture content based on the optical flow analysis results, the storage frame rate of the video data is reduced to a preset first frame rate. The video data is stored according to the preset first frame rate.

[0006] One possible implementation also includes: If the video data is determined to be video frame data with different picture content based on the optical flow analysis results, the storage frame rate of the video data is reduced to a preset second frame rate, wherein the preset first frame rate is less than the preset second frame rate. The video data is stored according to the preset second frame rate.

[0007] In one possible implementation, after storing the video data according to the preset first frame rate, the method further includes: Obtain the latest query time and preset duration range of the video data; The duration of the video data not being queried is determined based on the latest query time. The target preset duration range is determined based on the preset duration range in which the unqueried duration falls; Based on the correspondence between the preset duration range and the preset frame rate, a target preset frame rate corresponding to the target preset duration range is determined, wherein the target preset frame rate is less than the first frame rate; The video data is stored according to the target preset frame rate.

[0008] In one possible implementation, the preset duration range includes at least two duration ranges, and the longer the unqueried time, the smaller the preset frame rate of the corresponding preset duration range.

[0009] One possible implementation also includes: If the target importance score is greater than or equal to the preset score threshold, the video data is stored according to the original frame rate and original resolution of the video data, wherein the original frame rate is greater than the preset first frame rate.

[0010] In one possible implementation, after storing the video data according to its original frame rate and original resolution, the method further includes: Obtain the latest query time of the video data; The duration of the video data not being queried is determined based on the latest query time. If the unqueried duration exceeds a preset duration threshold, the video data is stored at a preset first resolution, which is less than the original resolution.

[0011] In one possible implementation, after reducing the storage frame rate of the video data to a preset first frame rate, the method further includes: Receive query commands sent by user terminal devices; The semantic information carried in the query instruction is searched in the locally stored structured data to obtain the target structured data; Based on the preset correspondence between structured data and video data, the target video data corresponding to the target structured data is determined; The target video data is reconstructed to obtain the reconstructed video data corresponding to the query command; The reconstructed video data and the semantic information in the target structured data are sent to the user terminal device so that the user terminal device can display the reconstructed video data and the semantic information in the target structured data on the display interface.

[0012] Secondly, embodiments of this application provide a video data storage apparatus applied to a cloud server, the apparatus comprising: The receiving module is used to receive video data and multimodal semantic information sent by the camera, wherein the multimodal semantic information is used to represent image features of different dimensions in the video data; The determination module is used to determine the target importance score corresponding to the target scene semantic information based on the target scene semantic information in the multimodal semantic information and according to the preset correspondence between scene semantic information and importance score; The optical flow analysis module is used to perform optical flow analysis on the video data to be stored when the target importance score is less than a preset score threshold, and obtain the optical flow analysis results. The reduction module is used to reduce the storage frame rate of the video data to a preset first frame rate when the video data is determined to be video frame data with the same picture content based on the optical flow analysis results. A storage module is used to store the video data according to the preset first frame rate.

[0013] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; A method for storing video data as described in the first aspect when the processor executes computer program instructions.

[0014] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the video data storage method as described in the first aspect is implemented.

[0015] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the video data storage method as described in the first aspect.

[0016] This application discloses a video data storage method, apparatus, device, storage medium, and program product. After receiving video data and corresponding multimodal semantic information from a camera, the cloud server extracts target scene semantic information from the multimodal semantic information. Based on a preset correspondence between scene semantic information and importance scores, it can determine whether important events exist in the video. Video frames that do not contain important events are stored at a low frame rate. Furthermore, if the target importance score is less than a preset threshold, image consistency analysis is performed on the video data that does not contain important events. This involves using optical flow analysis to determine whether the video data to be stored consists of video frames with identical image content, thereby further reducing the frame rate of such identical video frames. In this way, by judging the storage frame rate of video data layer by layer from two dimensions—whether the video data contains video data with important events and whether it consists of video frames with identical image content—compressible video data can be accurately identified, thereby reducing the storage space occupied by the video data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1a This is an exemplary schematic diagram of a method for storing video data in the prior art; Figure 1b This is an exemplary schematic diagram of a video data storage method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a video data storage method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating another video data storage method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating a video data query method provided in an embodiment of this application; Figure 5 This is an exemplary schematic diagram of another video data storage method provided in an embodiment of this application; Figure 6 This is an exemplary schematic diagram of a hierarchical decision-making frame extraction method provided in an embodiment of this application; Figure 7 This is an exemplary schematic diagram of a multi-scale downsampling method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a video data storage device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0021] like Figure 1a As shown, Figure 1a This example illustrates the process flow of a traditional video data storage method. In a traditional approach, after acquiring the surveillance video from the cloud server, a conventional frame extraction method is used to extract frames from the video data. Then, a conventional downsampling method is used to downsample the extracted video data. Upon receiving a query command from the user device, the corresponding video data is located locally, and then a conventional super-resolution interpolation method is used to reconstruct the video data to obtain the surveillance video. The judgment logic of the traditional video data storage method is relatively simple and cannot effectively identify the video data that needs to be compressed, resulting in the compressed video data still occupying a large amount of storage space.

[0022] To address the problems existing in the prior art, embodiments of this application provide a method, apparatus, device, storage medium, and program product for video data storage. Compared to traditional solutions, the solution provided in this application can simultaneously store structured data containing semantic information of the video data and compressed video data, reducing the storage space of the video data. Furthermore, the structured data enables rapid retrieval of video data. The structured data is used to store semantic information and preset modal attribute information of the video data. The preset modal attribute information can be related information such as timestamps, and the preset modality can be pre-set based on experience.

[0023] Figure 1b This embodiment of the present application provides a method for storing video data. After a camera captures surveillance video, it first performs quality enhancement processing on the video to obtain enhanced video. Then, it extracts multimodal semantic information from the enhanced video. The camera sends the multimodal semantic information and the enhanced video to a cloud server, which stores the multimodal semantic information as structured data, thereby establishing a correspondence between the structured data and the enhanced video. Furthermore, the cloud server performs hierarchical decision-based frame extraction on the enhanced video to determine the storage frame rate, and then performs multi-scale downsampling on the resolution of the extracted video to reduce the video resolution. After receiving a query command from a user device, the system searches the locally stored structured data for the semantic information carried in the query command to determine the corresponding video data. By reconstructing the video data, the surveillance video can be restored.

[0024] The following describes a video data storage method provided in an embodiment of this application, which is applied to a cloud server. For example... Figure 2 As shown, the method includes: S201, Receive video data and multimodal semantic information sent by the camera.

[0025] Multimodal semantic information is used to represent image features of different dimensions in video data. These different dimensional image features can be extracted using methods such as feature extraction algorithms, behavior recognition algorithms, and object detection algorithms. The specific image feature extraction algorithm corresponds to the type of image feature. For example, if an image feature represents the category of an object in a video image, then the extraction algorithm corresponding to that image feature is an object detection algorithm.

[0026] Specifically, to ensure the clarity and improve the quality of video data captured by the camera, the camera can perform quality enhancement processing on the captured video data. Quality enhancement methods include, but are not limited to, low-light scene optimization, rain removal, fog removal, and blur reduction.

[0027] In addition, the camera simultaneously sends video data timestamps and other preset attribute information to the cloud server. After receiving the multimodal semantic information and the multiple preset attribute information, the cloud server constructs the correspondence between the multimodal semantic information and the attribute information to obtain structured data. The preset dimensions are pre-set based on experience.

[0028] S202. Based on the target scene semantic information in the multimodal semantic information, determine the target importance score corresponding to the target scene semantic information according to the preset correspondence between scene semantic information and importance score.

[0029] Scene semantic information is used to represent events contained in the video data. For example, scene semantic information could be a child doing homework, an elderly person falling down, etc.

[0030] Different scene semantic information corresponds to different importance scores, and the preset correspondence between scene semantic information and importance scores is set in advance based on experience.

[0031] S203. When the target importance score is less than the preset score threshold, perform optical flow analysis on the video data to obtain the optical flow analysis results.

[0032] The preset score threshold is set based on experience. Optical flow analysis is used to estimate the motion direction and velocity of each pixel in an image sequence or video to obtain a motion vector, and then to determine whether the scene in the video has changed based on the motion vector.

[0033] S204. If the video data is determined to be video frame data with the same picture content based on the optical flow analysis results, reduce the storage frame rate of the video data to the preset first frame rate.

[0034] Specifically, the cloud storage server reduces the video data from the original frame rate to a preset first frame rate, where the original frame rate is the frame rate at which the camera captures video images.

[0035] S205. Store video data according to the preset first frame rate.

[0036] Using the above method, after receiving video data and corresponding multimodal semantic information from the camera, the cloud server extracts target scene semantic information from the multimodal semantic information. Based on the preset correspondence between scene semantic information and importance scores, it can determine whether there are important events in the video. Video frames that do not contain important events are stored at a low frame rate. Furthermore, if the target importance score is less than a preset threshold, image consistency analysis is performed on the video data that does not contain important events. This involves using optical flow analysis to determine whether the video data consists of video frames with identical content, thereby further reducing the frame rate of such identical video frames. In this way, by judging the storage frame rate of video data layer by layer from two dimensions—whether the video data contains video data with important events and whether the video frames consist of video frames with identical content—it is possible to accurately identify video data that can be compressed, thereby improving and reducing the storage space occupied by the video data.

[0037] In some embodiments of this application, the above-described video data compression method further includes: If the video data is determined to be video frame data with different picture content based on the optical flow analysis results, the storage frame rate of the video data is reduced to the preset second frame rate, and the video data is stored according to the preset second frame rate.

[0038] Among them, the preset first frame rate is less than the preset second frame rate, and the preset second frame rate is less than the original frame rate.

[0039] Using the method provided in the embodiments of this application, when the video data is determined to be video frame data with different picture content based on the optical flow analysis results, it indicates that the different picture content of the video data contains more information. Since the video data does not contain pictures of important events, the storage frame rate of the video data is reduced, and more information is preserved during the compression of the video data, thereby achieving accurate compression of the video data.

[0040] After compressing and storing video data on a cloud server, the importance of video data is highly time-sensitive. Recent video data is generally more relevant, so video data containing recent events is far more important than older video data. However, due to the long time span, older video data occupies significantly more storage space than video data containing recent events. For example, if a surveillance video hasn't been viewed within 7 days, the probability of it being viewed again is very low, and video data from the past six months occupies far more storage space than video data from the last 7 days. Therefore, older video data is further compressed by reducing the frame rate. Based on this, if... Figure 3 As shown, after storing video data according to a preset first frame rate in step S205, the method further includes: S301, Obtain the latest query time and preset duration range of video data.

[0041] The cloud server stores query records for each video data point, including the time of each query. The preset duration range is pre-set based on experience. In one example, the preset duration range could be 0-7 days, 8-30 days, or more than 30 days.

[0042] S302. Determine the duration of the video data that has not been queried based on the latest query time.

[0043] S303. Determine the target preset duration range based on the preset duration range in which the unqueried duration falls.

[0044] S304. Based on the correspondence between the preset duration range and the preset frame rate, determine the target preset frame rate corresponding to the target preset duration range.

[0045] The target preset frame rate is less than the preset first frame rate.

[0046] The correspondence between the preset duration range and the preset frame rate is preset based on experience.

[0047] In one example, the preset duration range is 0-7 days, and the corresponding preset frame rate can be the first preset frame rate. The preset duration range is 8-30 days, and the corresponding preset frame rate is the third preset frame rate, which is lower than the first preset frame rate. The preset duration range is more than 30 days, and the corresponding preset frame rate is the fourth preset frame rate, which is lower than the third preset frame rate.

[0048] S305. Store video data according to the target preset frame rate.

[0049] Using the method provided in this application, based on the query patterns of video data, the probability of video data being queried decreases as the duration of unqueried video data increases. Therefore, the frame rate of video data that has not been queried for a long time can be further reduced, thereby reducing the storage space occupied by the video data. Thus, storing video data at different frame rates for different preset duration ranges can effectively compress the storage space of the video data.

[0050] In the above Figure 3 Based on the illustrated embodiment, the storage space of video data can be further reduced by lowering the resolution. Therefore, the following describes the process of multi-scale downsampling of video images, which also includes: The preset duration range includes at least two duration ranges. The longer the unqueried time, the lower the preset frame rate of the corresponding preset duration range.

[0051] The correspondence between the preset duration range and the preset resolution is pre-set based on experience. In one example, the preset duration range is 0-7 days, and the corresponding preset frame rate is the first preset resolution, which is lower than the original resolution (the resolution at which the camera captures video data). The preset duration range is 8-30 days, and the corresponding preset resolution is the second preset resolution, which is lower than the first preset resolution. The preset duration range is more than 30 days, and the corresponding preset resolution is the third preset resolution, which is lower than the second preset resolution.

[0052] Using the method provided in this application, based on the query patterns of video data, the probability of video data being queried decreases as the duration of unqueried video data increases. Therefore, the resolution of video data that has not been queried for a long time can be further reduced, thereby reducing the storage space occupied by the video data. Thus, different resolutions are used to store video data for different preset duration ranges.

[0053] The following explanation addresses the case where the target importance score is greater than or equal to a preset score threshold. Specifically, when the target importance score is greater than or equal to the preset score threshold, the video data is stored according to the original frame rate and original resolution of the video data.

[0054] The original frame rate is greater than the preset first frame rate.

[0055] Understandably, when the target importance score is greater than or equal to the preset score threshold, it indicates that the video data contains video images of important events, that is, the video data contains a lot of information and has a high probability of being queried later. Therefore, in order to ensure the clarity and accuracy of the video images when users query, the cloud server uses the original frame rate and original resolution for storage.

[0056] Thus, by using target importance scoring to identify video images containing important events in the video data, the probability of this video data being queried later is high. In order to provide clearer video images in the query, the video data is stored according to the original frame rate and original resolution.

[0057] Understandably, after storing video data according to its original frame rate and resolution, the probability of that video data being retrieved decreases over time. This is because the resolution of the video data can be reduced to save storage space. Therefore, after storing video data according to its original frame rate and resolution, the method further includes: Step A: Obtain the latest query time for the video data.

[0058] The method for obtaining the latest query time is described in the above embodiments and will not be repeated here.

[0059] Step B: Determine the duration of the video data that has not been queried based on the latest query time.

[0060] Step C: If the query duration is not greater than the preset duration threshold, store the video data according to the preset first resolution.

[0061] The preset first resolution is smaller than the original resolution.

[0062] Understandably, a target importance score greater than a preset threshold indicates that the video data contains video images of important events. To ensure the clarity and accuracy of the video images, the cloud server stores them at a higher resolution and frame rate. Therefore, the video data is stored at a preset first resolution. In one example, the original resolution could be 3840×2160, while the preset first resolution is 1280×720, which is lower than the original resolution.

[0063] Using the method provided in this application, based on the query patterns of video data, the probability of video data being queried decreases as the duration of unqueried video data increases. Therefore, the resolution of video data that has not been queried for a long time can be further reduced, thereby reducing the storage space occupied by the video data. Thus, when the duration of unqueried video data exceeds a preset duration threshold, the original resolution is reduced to a preset first resolution. This saves storage space while ensuring the clarity of the video data.

[0064] After the video data storage is completed as described above, users can query the corresponding video data on the cloud server through their client devices. For example... Figure 4 As shown, the process of querying video data is as follows: S401, Receive query command sent by user terminal device.

[0065] The query command includes semantic information input by the user, which can be any type of multimodal semantic information. In one example, the semantic information input by the user is scene semantic information; for instance, the query command is to find video data containing images of an elderly person falling.

[0066] S402. Search for the semantic information carried in the query instruction in the locally stored structured data to obtain the target structured data.

[0067] Structured data is used to store multimodal semantic information of the video data. The camera synchronously sends timestamps and other preset attribute information of the video data to the cloud server. After receiving the multimodal semantic information and the preset attribute information, the cloud server constructs the correspondence between the multimodal semantic information and the attribute information to obtain structured data. The preset dimensions are pre-set based on experience.

[0068] S403. Based on the preset correspondence between structured data and video data, determine the target video data corresponding to the target structured data.

[0069] S404. Perform video reconstruction on the target video data to obtain the reconstructed video data corresponding to the query command.

[0070] The cloud server uses frame interpolation and super-resolution algorithms to reconstruct the target video data.

[0071] Specifically, after the cloud server reconstructs the video data, it corrects the reconstructed video based on the multimodal semantic information corresponding to the target video data to obtain reconstructed video data, thereby further improving the accuracy of the reconstructed video data.

[0072] S405. Send the reconstructed video data and semantic information in the target structured data to the user terminal device so that the user terminal device can display the semantic information in the reconstructed video data and target structured data on the display interface.

[0073] The cloud server can send reconstructed video data and semantic information of preset modalities in the target structured data to the user terminal device. The preset modalities are pre-set based on experience.

[0074] In one example, the preset modality may include scene semantic information, target category semantic information, timestamp information, and target location semantic information.

[0075] In addition, after receiving the semantic information of the preset modality in the reconstructed video data and the target structured data, the user terminal device displays the reconstructed video data on the display interface. In response to the operation of the mouse cursor on the reconstructed video data, the aforementioned semantic information of the preset modality is displayed on a different layer than the reconstructed video data.

[0076] Using the method provided in this application, after the cloud server receives a query command sent by the user terminal device, it can search for target structured data containing the same semantic information in the locally stored structured data according to the semantic information carried in the query command. Based on the preset correspondence between structured data and video data, the compressed and stored video data corresponding to the target structured data can be determined. By reconstructing the found compressed and stored video data, the video data queried by the user can be obtained, thus improving data search efficiency.

[0077] like Figure 5 As shown, Figure 5 An exemplary method for storing video data is shown, the method comprising: S501: The camera enhances the quality of surveillance video and extracts multimodal semantic information.

[0078] The methods for quality enhancement and multimodal semantic information extraction are described in the above embodiments and will not be repeated here.

[0079] S502, the cloud server performs adaptive hierarchical decision-making for frame extraction of surveillance video.

[0080] In this context, the adaptive hierarchical decision frame extraction is the process described in the above embodiments of judging the frame rate of video data layer by layer based on three dimensions: whether it contains important events, whether the screen content is consistent, and whether it has not been queried for a long time.

[0081] S503, perform multi-scale downsampling on the intra-frame resolution.

[0082] Multi-scale downsampling refers to the process described in the above embodiments of determining the resolution of video data layer by layer based on two dimensions: whether it contains important events and whether it has not been queried for a long time.

[0083] S504. Reconstruct and restore the video data retrieved by the user.

[0084] The method provided in this application, after quality enhancement processing of the surveillance video, extracts multimodal semantic information, then determines the storage frame rate of the surveillance video through hierarchical decision frame extraction, and determines the storage resolution of the surveillance video through multi-scale downsampling, thereby reducing the storage space of the video data. After the cloud server receives the query command sent by the user terminal device, it restores and reconstructs the found video data to obtain the reconstructed video data, thus improving the efficiency of video data reconstruction.

[0085] The following combination Figure 6 The above-mentioned adaptive hierarchical decision frame extraction method is introduced, such as... Figure 6 As shown, the method includes: S601, acquire high frame rate surveillance video.

[0086] High frame rate surveillance video refers to video captured by cameras. High frame rate means the original frame rate of the surveillance video.

[0087] S602. Determine whether important events are included.

[0088] If yes, then execute S603; otherwise, execute S604.

[0089] Specifically, an importance score for the surveillance video is calculated based on the scene semantic information corresponding to the video. Then, the importance score and a preset score threshold are used to determine whether the video contains important events. For specific methods, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0090] S603, stored as high frame rate surveillance video.

[0091] S604, stored as medium frame rate monitoring video.

[0092] S605. Determine if the content of the screen is consistent.

[0093] If yes, then execute S606; otherwise, execute S604.

[0094] S606, stored as low frame rate surveillance video.

[0095] S607. Determine if the document has not been viewed for a long time.

[0096] If yes, then execute S608; otherwise, execute S606.

[0097] Specifically, the method for determining the target preset frame rate based on the unviewed duration is described in the relevant description in the above embodiments, and will not be repeated here.

[0098] S608, stored as surveillance video at extremely low frame rates.

[0099] The method provided in this application determines the storage frame rate of the surveillance video layer by layer based on three dimensions: whether the video data contains information about important events, whether the content of the video data is relatively static, and whether it has not been viewed for a long time. This effectively identifies the data in the surveillance video that can be compressed, thereby effectively reducing the storage space of the video data.

[0100] The following combination Figure 7 The above-mentioned multi-scale downsampling methods are introduced, such as... Figure 7 As shown, the method includes: S701, acquire high-resolution surveillance video.

[0101] S702. Determine whether important events are included.

[0102] If yes, then execute S708; otherwise, execute S703.

[0103] S703, stored as medium resolution video.

[0104] S704. Determine if the device has not been viewed for 7 consecutive days.

[0105] If yes, then execute S705; otherwise, execute S703.

[0106] S705, store as low-resolution video.

[0107] S706. Determine if the document has not been viewed for 30 consecutive days.

[0108] If yes, then execute S707; otherwise, execute S705.

[0109] S707, stored as extremely low resolution video.

[0110] S708, stored as high-resolution video.

[0111] S709. Determine if the device has not been viewed for 7 consecutive days.

[0112] If yes, then execute S710; otherwise, execute S708.

[0113] S710, stored as medium resolution video.

[0114] The method provided in this application determines the storage resolution of surveillance video by whether it contains video data of important events, whether it has not been viewed for 7 consecutive days, and whether it has not been viewed for 30 consecutive days. This effectively identifies data in the surveillance video that can be compressed, thereby effectively reducing the storage space occupied by the video data.

[0115] Based on the same concept, embodiments of this application provide a video data storage device applied to a cloud server, such as... Figure 8 As shown, the device includes: The receiving module 801 is used to receive video data and multimodal semantic information sent by the camera, wherein the multimodal semantic information is used to represent image features of different dimensions in the video data; The determining module 802 is used to determine the target importance score corresponding to the target scene semantic information based on the target scene semantic information in the multimodal semantic information and according to the preset correspondence between scene semantic information and importance score; The optical flow analysis module 803 is used to perform optical flow analysis on the video data when the target importance score is less than a preset score threshold, and obtain the optical flow analysis result. The reduction module 804 is used to reduce the storage frame rate of the video data to a preset first frame rate when the video data is determined to be video frame data with the same picture content based on the optical flow analysis result. Storage module 805 is used to store the video data according to the preset first frame rate.

[0116] In one possible implementation, the reduction module 804 is further configured to reduce the storage frame rate of the video data to a preset second frame rate when the video data is determined to be video frame data with different picture content based on the optical flow analysis result, wherein the preset first frame rate is less than the preset second frame rate. The storage module 805 is also used to store the video data according to the preset second frame rate.

[0117] In one possible implementation, the device further includes: The acquisition module is used to acquire the latest query time and preset duration range of the video data; The determining module 802 is further configured to determine the unqueried duration of the video data based on the latest query time; The determining module 802 is further configured to determine a target preset duration range based on the preset duration range in which the unqueried duration is located; The determining module 802 is further configured to determine the target preset frame rate corresponding to the target preset duration range based on the correspondence between the preset duration range and the preset frame rate, wherein the target preset frame rate is less than the first frame rate; The storage module 805 is also used to store the video data according to the target preset frame rate.

[0118] In one possible implementation, the preset duration range includes at least two duration ranges, and the longer the unqueried time, the smaller the preset frame rate of the corresponding preset duration range.

[0119] In one possible implementation, the storage module 805 is further configured to store the video data according to the original frame rate and original resolution of the video data when the target importance score is greater than or equal to the preset score threshold, wherein the original frame rate is greater than the preset first frame rate.

[0120] In one possible implementation, the acquisition module is further configured to acquire the latest query time of the video data; The determining module 802 is further configured to determine the unqueried duration of the video data based on the latest query time; Storage module 805 is further configured to store the video data according to the preset first resolution when the unqueried duration is greater than the preset duration threshold, wherein the preset first resolution is less than the original resolution.

[0121] In one possible implementation, the device further includes: The receiving module 801 is used to receive query commands sent by the user terminal device; The search module is used to search for the semantic information carried in the query instruction in the locally stored structured data to obtain the target structured data; The determining module 802 is further configured to determine the target video data corresponding to the target structured data based on a preset correspondence between the structured data and the video data; The reconstruction module is used to reconstruct the target video data to obtain the reconstructed video data corresponding to the query command. The sending module is used to send the reconstructed video data and the semantic information in the target structured data to the user terminal device, so that the user terminal device can display the reconstructed video data and the semantic information in the target structured data on the display interface.

[0122] It should be noted that the device for storing video data is the same as the method for storing video data described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0123] Figure 9 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0124] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0125] Specifically, the processor 901 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0126] Memory 902 may include mass storage for data or instructions. For example, and not limitingly, memory 902 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 902 may include removable or non-removable (or fixed) media. Where appropriate, memory 902 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 902 is non-volatile solid-state memory.

[0127] In a particular embodiment, memory 902 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0128] The processor 901 implements any of the video data storage methods described in the above embodiments by reading and executing computer program instructions stored in the memory 902.

[0129] In one example, the electronic device may also include a communication interface 903 and a bus 904. For example, Figure 9 As shown, the processor 901, memory 902, and communication interface 903 are connected through bus 904 and complete communication with each other.

[0130] The communication interface 903 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0131] Bus 904 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Super Transmission (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 904 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0132] Furthermore, in conjunction with the video data storage methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the video data storage methods described in the above embodiments.

[0133] This application also provides a computer program product, including a computer program, which, when executed, implements any of the video data storage methods described in the above embodiments.

[0134] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0135] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on machine-readable media or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0136] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0137] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0138] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for storing video data, characterized in that, Applied to cloud servers, the method includes: The system receives video data and multimodal semantic information transmitted from a camera, wherein the multimodal semantic information is used to represent image features of different dimensions in the video data. Based on the target scene semantic information in the multimodal semantic information, and according to the preset correspondence between scene semantic information and importance score, the target importance score corresponding to the target scene semantic information is determined; If the target importance score is less than a preset score threshold, optical flow analysis is performed on the video data to obtain the optical flow analysis results. If the video data is determined to be video frame data with the same picture content based on the optical flow analysis results, the storage frame rate of the video data is reduced to a preset first frame rate. The video data is stored according to the preset first frame rate.

2. The method according to claim 1, characterized in that, Also includes: If the video data is determined to be video frame data with different picture content based on the optical flow analysis results, the storage frame rate of the video data is reduced to a preset second frame rate, wherein the preset first frame rate is less than the preset second frame rate. The video data is stored according to the preset second frame rate.

3. The method according to claim 1, characterized in that, After storing the video data according to the preset first frame rate, the method further includes: Obtain the latest query time and preset duration range of the video data; The duration of the video data not being queried is determined based on the latest query time. The target preset duration range is determined based on the preset duration range in which the unqueried duration falls; Based on the correspondence between the preset duration range and the preset frame rate, a target preset frame rate corresponding to the target preset duration range is determined, wherein the target preset frame rate is less than the first frame rate; The video data is stored according to the target preset frame rate.

4. The method according to claim 3, characterized in that, The preset duration range includes at least two duration ranges, and the longer the unqueried duration, the smaller the preset frame rate of the corresponding preset duration range.

5. The method according to claim 1, characterized in that, Also includes: If the target importance score is greater than or equal to the preset score threshold, the video data is stored according to the original frame rate and original resolution of the video data, wherein the original frame rate is greater than the preset first frame rate.

6. The method according to claim 5, characterized in that, After storing the video data according to its original frame rate and original resolution, the method further includes: Obtain the latest query time of the video data; The duration of the video data not being queried is determined based on the latest query time. If the unqueried duration exceeds a preset duration threshold, the video data is stored at a preset first resolution, which is less than the original resolution.

7. The method according to claim 1, characterized in that, After reducing the storage frame rate of the video data to a preset first frame rate, the method further includes: Receive query commands sent by user terminal devices; The semantic information carried in the query instruction is searched in the locally stored structured data to obtain the target structured data, which includes the multimodal semantic information corresponding to the video data. Based on the preset correspondence between structured data and video data, the target video data corresponding to the target structured data is determined; The target video data is reconstructed to obtain the reconstructed video data corresponding to the query command; The reconstructed video data and the semantic information in the target structured data are sent to the user terminal device so that the user terminal device can display the reconstructed video data and the semantic information in the target structured data on the display interface.

8. A device for storing video data, characterized in that, The device, applied to a cloud server, includes: The receiving module is used to receive video data and multimodal semantic information sent by the camera, wherein the multimodal semantic information is used to represent image features of different dimensions in the video data; The determination module is used to determine the target importance score corresponding to the target scene semantic information based on the target scene semantic information in the multimodal semantic information and according to the preset correspondence between scene semantic information and importance score; The optical flow analysis module is used to perform optical flow analysis on the video data when the target importance score is less than a preset score threshold, and obtain the optical flow analysis results. The reduction module is used to reduce the storage frame rate of the video data to a preset first frame rate when the video data is determined to be video frame data with the same picture content based on the optical flow analysis results. A storage module is used to store the video data according to the preset first frame rate.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; The processor executes computer program instructions to implement the video data storage method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the video data storage method as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the video data storage method as described in any one of claims 1-7.

Images