Video generation method, device, system, equipment and medium

Abstract

The present application discloses a video generation method, device, system, equipment and medium, which are used to generate videos that better meet the needs of users, avoid the acquisition and storage of video materials that users do not need, save storage space, improve the generation efficiency and effect of videos required by users, and improve user experience. The method provided by the present application includes: determining algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the demand information pre-entered by the user, and the demand information is information about the video to be generated; based on the algorithm startup parameters, start the preset algorithm to detect the collected video; if the detection result for the current frame image is consistent with the demand information pre-entered by the user, then starting from the current frame image, determine at least one video material; wherein the at least one video material is used to generate the video required by the user.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a video generation method, device, system, equipment and medium. Background Art

[0002] In recent years, the short video industry has become popular, and people need more video materials to share their daily lives. Security surveillance cameras will continuously generate video materials during operation, but the generated video data is relatively large, and it is difficult for people to find the videos they want. After finding the video materials, they still need to perform complicated video editing. Video editing is highly professional, requires a lot of time, and is difficult for non-professionals to operate. Summary of the invention

[0003] The embodiments of the present application provide a video generation method, apparatus, system, equipment and medium for generating videos that better meet the needs of users, avoiding the acquisition and storage of video materials that users do not need, saving storage space, and improving the generation efficiency and effect of videos required by users, thereby improving the user experience.

[0004] A video generation method provided in an embodiment of the present application includes:

[0005] Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated;

[0006] Based on the algorithm startup parameters, start the preset algorithm to detect the collected video;

[0007] If the detection result of the current frame image is consistent with the requirement information pre-input by the user, at least one video material is determined starting from the current frame image; wherein the at least one video material is used to generate the video required by the user.

[0008] The method provided in the embodiment of the present application determines the algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the demand information pre-input by the user, and the demand information is information about the video to be generated, so that the preset algorithm can be started based on the algorithm startup parameters to perform more targeted and accurate video detection on the collected video; then, if the detection result for the current frame image is consistent with the demand information pre-input by the user, then starting from the current frame image, at least one video material is determined, so that the determined video material is richer and it is not easy to miss every wonderful shot; wherein the at least one video material is used to generate the video required by the user. Therefore, the method adopted in the embodiment of the present application can generate a video that better meets the user's needs for the user, avoids the acquisition and storage of video materials that the user does not need, saves storage space, and improves the generation efficiency and effect of the video required by the user, thereby improving the user experience.

[0009] In some embodiments, starting from the current frame image, determining at least one segment of video material includes:

[0010] The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

[0011] In some embodiments, the method further comprises:

[0012] In the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user.

[0013] In some embodiments, determining statistical data includes:

[0014] When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material;

[0015] If new video material collection is not started, then:

[0016] If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table;

[0017] If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table;

[0018] If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

[0019] In some embodiments, the method further comprises:

[0020] Based on the at least one video material, screening the video material;

[0021] The screened video material is sent to the cloud, and the video required by the user is generated through the cloud.

[0022] In some embodiments, based on the at least one video material, performing video material screening includes:

[0023] For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

[0024] In some embodiments, based on the at least one video material, performing video material screening includes:

[0025] For video materials of multiple days, the video materials are classified according to a single day to obtain classification results;

[0026] For a single day's video material, the video material is classified according to time to obtain a classification result;

[0027] For each of the above classification results:

[0028] If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing;

[0029] If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold.

[0030] Another video generation method provided in an embodiment of the present application includes:

[0031] Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated;

[0032] Sending the algorithm startup parameters to the target camera;

[0033] Receiving at least one video material obtained by the target camera based on the algorithm startup parameters;

[0034] A video required by a user is generated based on the at least one video material.

[0035] In some embodiments, determining algorithm startup parameters for detecting the captured video includes:

[0036] Acquire the demand information input by the user;

[0037] Performing semantic analysis on the demand information to obtain analysis results;

[0038] If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determining the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, outputting user prompt information;

[0039] If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, then the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, then user prompt information is output.

[0040] An embodiment of the present application provides a video generation device, including:

[0041] An algorithm startup parameter determination unit, used to determine an algorithm startup parameter for detecting the collected video; wherein the algorithm startup parameter is determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated;

[0042] A video real-time detection unit, used to start a preset algorithm to detect the collected video based on the algorithm startup parameters;

[0043] The video material determination unit is used to determine at least one video material starting from the current frame image if the detection result for the current frame image is consistent with the requirement information pre-input by the user; wherein the at least one video material is used to generate the video required by the user.

[0044] Another video generation device provided in an embodiment of the present application includes:

[0045] An algorithm startup parameter determination unit, used to determine an algorithm startup parameter for detecting the collected video; wherein the algorithm startup parameter is determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated;

[0046] An algorithm startup parameter sending unit, used to send the algorithm startup parameters to a target camera;

[0047] A video material acquisition unit, configured to receive at least one video material obtained by the target camera based on the algorithm startup parameters;

[0048] The video synthesis unit is used to generate a video required by a user based on the at least one video material.

[0049] A video generation system provided in an embodiment of the present application includes: a cloud device and a camera;

[0050] The cloud device is used to determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the demand information pre-entered by the user, and the demand information is information about the video to be generated; the algorithm startup parameters are sent to the camera; at least one video material obtained by the camera based on the algorithm startup parameters is received; and the video required by the user is generated based on the at least one video material;

[0051] The camera is used to receive algorithm startup parameters sent by the cloud device for detecting the collected video; based on the algorithm startup parameters, start a preset algorithm to detect the collected video; if the detection result of the current frame image is consistent with the requirement information pre-entered by the user, then starting from the current frame image, determine at least one video material; and send the at least one video material to the cloud device.

[0052] Another embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the above methods according to the obtained program.

[0053] Another embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A schematic diagram of a flow chart of a video generation method provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of a method for collecting video material with a camera provided in an embodiment of the present application;

[0057] Figure 3 A schematic flow chart of a method for determining statistical data of text information by a camera provided in an embodiment of the present application;

[0058] Figure 4 A schematic flow chart of a method for selecting video materials using a camera provided in an embodiment of the present application;

[0059] Figure 5 A schematic flow chart of another method for selecting video materials by a camera provided in an embodiment of the present application;

[0060] Figure 6 A flowchart of another video generation method provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of a method flow for determining algorithm startup parameters for a cloud device provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0063] Fig. 9 A schematic diagram of the structure of a video generating device provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of the structure of another video generating device provided in an embodiment of the present application;

[0065] Fig.11 A schematic diagram of the structure of a video generation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] The embodiments of the present application provide a video generation method, apparatus, system, equipment and medium for generating videos that better meet the needs of users, avoiding the acquisition and storage of video materials that users do not need, saving storage space, and improving the generation efficiency and effect of videos required by users, thereby improving the user experience.

[0068] Among them, the method and the device, system, equipment, and medium are based on the same application concept. Since the principles of solving problems by the method and the device, system, equipment, and medium are similar, the implementation of the device, system, equipment, medium, and method can refer to each other, and the repeated parts will not be repeated.

[0069] The terms "first", "second", etc. (if any) in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0070] The following examples and embodiments are to be understood as illustrative examples only. Although this specification may refer to "one", "an" or "some" examples or embodiments in several places, this does not mean that each such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. In addition, terms such as "including" and "comprising" should be understood as not limiting the described embodiments to consist only of those features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that are not specifically mentioned.

[0071] The following is a detailed description of each embodiment of the present application in conjunction with the accompanying drawings. It should be noted that the display order of the embodiments of the present application only represents the order of the embodiments, and does not represent the advantages and disadvantages of the technical solutions provided by the embodiments. Example

[0072] See also Figure 1 , a video generation method provided in an embodiment of the present application, for example, can be applied on the camera side, the method comprising:

[0073] S101, determining algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated;

[0074] In some embodiments, the algorithm startup parameters for detecting the collected video may be determined by the camera obtaining the algorithm startup parameters for detecting the collected video from the cloud device based on the demand information pre-input by the user; of course, the algorithm startup parameters for detecting the collected video may also be determined by the camera locally based on the demand information pre-input by the user.

[0075] In some embodiments, the algorithm startup parameters include, for example, the video detection algorithm that needs to be started, the detection area on the image, the detection sensitivity, the indication information of whether to start voice detection, etc.

[0076] In some embodiments, the demand information is obtained by, for example, providing a user interface pre-developed on a web or APP, with a theme text input window, a time limit drop-down window, a video length setting window, etc. for the video that the user needs to generate, thereby obtaining the demand information input by the user.

[0077] Among them, in the theme text input window, the user can input the theme of the video he wants to generate through text.

[0078] In the time limit drop-down window, the user can select the video generation time limit through the drop-down window, and the selection range is 1-30 days, that is, videos within the next 1-30 days need to be screened for materials.

[0079] In the video duration setting window, users can set the duration of the generated video.

[0080] In some embodiments, after the user inputs the demand information through the client, the demand information can be sent to the cloud. After receiving the demand information, the cloud runs a natural language processing algorithm (including but not limited to an NLP algorithm) to perform semantic analysis on the input topic and extract key features, such as the characters, behaviors, places, and times in the topic. The characters can be people, animals, plants, or objects.

[0081] For example, if a user wants to see what their pet dog and cat have done today after get off work, they can select a one-day time limit through the app before going to work, and the theme is: A day for dogs and cats, record the activities of dogs and cats today, etc. For another example, if a user wants to record the dinner time during the Spring Festival, he will enter the theme through the app before the Spring Festival: Please record the video of our family entertainment in the next three days. No matter how the user enters, the natural language processing algorithm will extract the key features of the theme.

[0082] S102, based on the algorithm startup parameters, starting a preset algorithm to detect the collected video;

[0083] The preset algorithm, i.e., the algorithm used for video detection, may be, for example, a face detection algorithm, a behavior detection algorithm, etc. In the embodiment of the present application, a plurality of target detection algorithms may be preset to realize target detection for different video contents according to different needs of users. Thus, in practical applications, the corresponding algorithm may be selected for video detection according to the demand information pre-entered by the user.

[0084] S103. If the detection result of the current frame image is consistent with the requirement information pre-input by the user, then starting from the current frame image, at least one video material is determined; wherein the at least one video material is used to generate the video required by the user.

[0085] The detection result for the current frame image is consistent with the demand information pre-entered by the user. For example, if the preset algorithm started detects a dog from the current frame image, and the subject input by the user also includes a dog, then it is considered that the detection result for the current frame image is consistent with the demand information pre-entered by the user. Then, correspondingly, starting from the current frame image, at least one video material is determined, that is, when a dog appears in the video image, the video material is generated. A video material of a preset length can be generated, or multiple video materials of a preset length can be generated. Among them, the preset length can be a default length, or a length entered by the user (included in the demand information pre-entered by the user).

[0086] In some embodiments, starting from the current frame image, determining at least one segment of video material includes:

[0087] The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

[0088] For example, see Figure 2 The method flow of collecting video material by a camera provided in the embodiment of the present application includes:

[0089] S201, obtaining a current frame image collected in real time.

[0090] S202: Determine the detection result of the current frame image.

[0091] That is to say, the current frame image collected in real time is sent to a preset algorithm for detection, thereby obtaining the detection result of the current frame image.

[0092] For example, after the camera starts the preset algorithm according to the configuration of the algorithm startup parameters, the real-time collected video frames are sent to the algorithm for real-time detection and analysis. If the detected target and target behavior meet the required information input by the user (for example, the target and target behavior appearing in the user input subject), the detection result is analyzed.

[0093] Since the user may input multiple targets and target behaviors, the results returned by the algorithm will also include multiple characters, each of which will have corresponding behaviors. Therefore, in some embodiments, the behavior coordinates of the character can also be output, that is, the detection results can also include the behavior coordinates of the character.

[0094] In some embodiments, if the analysis is performed on a person, the person's emotions may also be identified based on voice detection information and facial recognition, that is, the detection result may also include the person's emotional information, etc.

[0095] S203, determine whether the detection result of the current frame image is consistent with the required information pre-entered by the user, if yes, execute step S204; otherwise, execute step S201 to continue to obtain the current frame image collected in real time.

[0096] That is to say, if for each frame image in the video, the detection result output by the algorithm meets the requirement information pre-entered by the user (that is, meets the requirements regarding the video material), for example, the recognized behavior result is consistent with the behavior parsed from the subject input by the user, then the current frame image is considered to meet the requirement information pre-entered by the user.

[0097] When the real-time captured image meets the user's needs, the video material is collected from the current frame. The video material contains timestamp information. Because the behavior is continuous, when the material collection starts, the system will maintain a comparison frame. The time difference between the current frame and the comparison frame is used to determine whether the material collection is finished. If the time difference is greater than the threshold, the current material collection is considered to be completed and a new material collection is started. That is, for each video material collection, if the time difference between the start time and the current time is less than the threshold, it is regarded as the same video material. If the time difference is greater than or equal to the threshold, a new material collection is started.

[0098] S204, determine whether the comparison frame is empty, if so, execute step S208; otherwise, execute step S205.

[0099] If the comparison frame is empty, it means that it is necessary to start collecting new video materials, so step S208 is executed; otherwise, if there is a comparison frame, it means that the collection of the current video material has not been completed and new images need to be added, so step S205 is executed.

[0100] It can be seen that the role of the comparison frame described in the embodiment of the present application is to control the duration of each video material.

[0101] S205: Determine the time difference between the timestamp of the current frame image and the time stamp of the comparison frame.

[0102] S206, determining whether the time difference between the timestamp of the current frame image and the time stamp of the comparison frame is greater than a preset threshold, if yes, executing step S208; otherwise, executing step S207.

[0103] S207: Count the current frame image into the current segment of video material, and use the current frame as a comparison frame.

[0104] S208: Start collecting new video material, add the current frame image to the new video material, and use the current frame as a comparison frame.

[0105] The current frame is used as the comparison frame, that is, the time value of the comparison frame is updated to be the timestamp of the current frame image.

[0106] In some embodiments, the method further comprises:

[0107] In the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user.

[0108] The statistical data include, for example, characters, behaviors, and the frequency, duration, and coordinates of the behaviors.

[0109] For example, during the video material collection process, the camera will maintain a statistical data for generating text information based on the video material, in preparation for the subsequent generation of text when the cloud device synthesizes the video. The statistical data describing the text information mainly includes recording people, behaviors, and the frequency, duration, coordinates, etc. of the behavior.

[0110] The duration is the duration of the character's behavior; the frequency is the number of times the character's behavior occurs after the statistical data begins.

[0111] In some embodiments, if the detection target is a person, the preset algorithm activated may be a face detection algorithm, so that facial expression recognition may also be performed, and emotional information may be recorded into statistical data based on the facial expression recognition results.

[0112] In some embodiments, if the target is an object, the transformation process of the object will also be recorded.

[0113] For example, the transformation process of an object is based on time, and the transformation process is recorded by comparing image brightness and time. For example, if the camera's field of view is always facing a pot of flowers, then key time points such as morning, noon and evening are selected for recording. The image brightness at different time points will be different, and the point with the largest brightness difference in the current time period is used as the screening video to reflect the transformation of the object, so that the screened video has a more layered feel.

[0114] In some embodiments, in order to maintain the frequency information of character behaviors, a global character behavior statistical frequency table can be maintained separately. When new materials are generated, the table is checked for comparison, and one is added to the existing character behavior frequency to obtain the frequency information of the same behavior of the same character. In other words, the global character behavior statistical frequency table is used to record the number of times each behavior of each character appears.

[0115] In some embodiments, determining statistical data includes:

[0116] When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material;

[0117] If the new video material collection has not started (that is, the current frame image is not the first frame image of the new video material), then:

[0118] If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table;

[0119] If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table;

[0120] If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

[0121] For example, see Figure 3 , methods for determining statistics include:

[0122] S301, obtaining a current frame image collected in real time.

[0123] S302: Determine whether new video material collection is currently started. If yes, execute step S303; otherwise, execute step S305.

[0124] That is to say, if the current frame image captured in real time is used as the first frame image of the new video material, step S303 needs to be executed; otherwise, step S305 is executed.

[0125] S303: Record the behavior, start frame identifier, and coordinate information of the person in the current frame image.

[0126] S304: Record the behavior frequency of the character according to the maintained global character behavior statistical frequency table.

[0127] S305, determining whether a new behavior occurs in the current frame image, if so, executing step S303; otherwise, executing step S305.

[0128] S306: Record the start frame identifier of the new behavior.

[0129] S307: Record the end frame identifier of the previous behavior.

[0130] That is to say, for the same target, when a new behavior occurs, not only the start frame identifier of the target's new behavior is recorded, but also the end frame identifier of the target's previous behavior is recorded. For example, if the current frame is the nth frame, and for the same person, a new behavior occurs, the start frame of the new behavior is recorded as the nth frame, and the end frame identifier of the person's previous behavior is the n-1th frame.

[0131] S308. Record the character's new behavior frequency according to the maintained global character behavior statistical frequency table.

[0132] S309, determine whether a new person appears in the current frame image, if yes, execute step S303; otherwise, execute step S305.

[0133] S310, recording the new character, the behavior of the new character, the start frame identifier, and the coordinate information.

[0134] S311. Record the behavior frequency of the new character according to the maintained global character behavior statistical frequency table.

[0135] S312, determine whether any person disappears in the current frame image, if yes, execute step S303; otherwise, execute step S305.

[0136] S313. Determine the missing person.

[0137] S314: Record the end frame mark of the disappeared person.

[0138] S315: Update the end frame identifier of the statistical data.

[0139] For example, two characters appear in a video material, the first character has two behaviors, and the second character has two behaviors. Then, for each behavior of each character, a corresponding statistical data will be generated through the embodiment of the present application.

[0140] In some embodiments, the method further comprises:

[0141] Based on the at least one video material, screening the video material;

[0142] The screened video material is sent to the cloud, and the video required by the user is generated through the cloud.

[0143] In the related art, generating a video requires storing a large amount of video material for screening, which easily causes a large amount of useless material to be stored, resulting in a waste of resources. Therefore, in the embodiment of the present application, when the camera captures the video, the video material is screened, so that the useless material can be eliminated at the first time, saving storage resources.

[0144] The video material screening is performed based on the at least one video material, for example, in the following two ways:

[0145] Method 1: In some embodiments, based on the at least one video material, video material screening is performed, including:

[0146] For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

[0147] The video quality is relatively high, for example, a comprehensive evaluation result obtained by evaluating the audio and image quality of the video is relatively high. How to evaluate the video quality may be determined according to actual needs.

[0148] Method 2: In some embodiments, based on the at least one video material, video material screening is performed, including:

[0149] For video materials from multiple days, the video materials are classified according to the single day to obtain the classification results (i.e., each day corresponds to one video);

[0150] For a single day's video material, the video material is classified according to time to obtain a classification result (for example, the single day's video material is evenly divided into four video segments);

[0151] For each of the above classification results:

[0152] If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing;

[0153] If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold.

[0154] The above two methods can be used alone or in combination.

[0155] That is to say, during the process of collecting video materials, the total length of the collected materials may be greater than the total video length set by the user. At this time, the materials need to be screened. If the video length is still too long after the screening is completed, the materials will be synthesized into multiple video outputs that meet the length requirements.

[0156] The following two situations need to be considered when screening materials:

[0157] Scenario 1: The target cycles in and out of the monitoring view;

[0158] Situation 2: The target is always under surveillance.

[0159] In the first case, the algorithm detects that the target video does not have long-term continuity, there will be more sliced ​​materials, and there may be multiple segments of materials with a large number of repeated behaviors.

[0160] In the second case, the video has a long duration. There may be a situation where a certain behavior in the material lasts for a long time, and there is a single behavior material type in the material.

[0161] Then, for the first case, there are multiple video segments v1, v2, v3, ..., Vn. If all the multiple segment materials are synthesized, the video will be too long. Therefore, in some embodiments, see Figure 4 , you can use the following method (corresponding to the above method 1) to delete redundant materials:

[0162] S401: For every two video materials, compare the two video materials.

[0163] S402, determine whether the characters in the two video materials are the same and whether the character behaviors are also the same. If so, execute step S404; otherwise, that is, the characters and / or character behaviors in the two video materials are different, execute step S403.

[0164] S403. Keep these two video clips.

[0165] S404: Compare the image quality of the two video clips.

[0166] S405. Keep video material with higher image quality.

[0167] That is to say, if the characters and behaviors in two materials are exactly the same, the video materials are evaluated for video quality. For example, the video quality evaluation can be combined with the quality of audio and image to achieve higher accuracy. For example, a weighted comparison can be made from multiple dimensions such as image brightness, contrast, saturation, noise information, and video quantization parameters to distinguish the quality of the video, and delete the material with a lower video quality score. However, when deleting the material, the statistical data of the text information needs to be updated, and the text information of the deleted material needs to be maintained. That is to say, the video material is deleted, but the statistical data of its text information can be retained, thereby keeping the total frequency information of the global character behavior unchanged.

[0168] For the second case, the target is always under the monitoring view. In this case, there is a situation where a certain behavior lasts for a long time. For this situation (corresponding to the above method 2), see Figure 5 , you can use the following methods to filter video materials:

[0169] S501, classifying video materials according to time to obtain multiple classification results;

[0170] That is to say, for multi-day video materials, the video materials are classified according to single days to obtain classification results, that is, for multi-day materials, the materials are classified according to single days; for single-day video materials, the video materials are classified according to time to obtain classification results, for example, they are divided into four sections according to time to ensure the differences of the materials;

[0171] S502, obtaining the current classification result;

[0172] S503, determining whether the character behaviors in the current classification result are the same behavior, if yes, executing step S505; otherwise, executing step S504;

[0173] S504, slicing the current classification result according to different behaviors to obtain multiple slices; and performing frame extraction processing on the slices whose behavior duration exceeds a preset threshold;

[0174] Among them, each slice corresponds to a video of a behavior;

[0175] S505: determine whether the duration of the same behavior exceeds a preset threshold. If yes, execute step S506; otherwise, return to step S502 to continue to obtain the next classification result and perform subsequent judgment processing steps, and so on, until all classification results have completed the judgment of whether frame extraction processing is required;

[0176] S506: Perform frame extraction processing on the current classification result. The frame extraction processing, for example, finds the key frames of the video, retains them, and removes non-key frames.

[0177] It can be seen that after classifying the video materials, materials are taken from each category for frame extraction. The video materials that need frame extraction have the following characteristics: the video contains the same behavior for a long time. If there is no same behavior, the current classification result is further sliced ​​according to different behaviors, and the duration of each behavior is judged. If the behavior duration exceeds the preset threshold, or the proportion of the behavior in the current classification result is higher than the preset threshold, the video slice corresponding to the behavior is subjected to frame extraction.

[0178] In summary, in order to avoid the lengthy and poor quality of the generated video, the embodiment of the present application provides a screening scheme for video materials, which uses some physical parameters of the camera during acquisition to screen images, and removes duplicate materials through classification and video quality measurement methods before synthesizing the video. The video quality measurement will take into account the image attributes, brightness, clarity, and audio detection results of the video within the detection range, and judge the quality of the video by weight to select the video clips. At the same time, by retaining the key frame method, the material with a long single material time is framed, which avoids the lengthy generated video. Example

[0179] See also Figure 6 Another video generation method provided in an embodiment of the present application, for example, can be applied to a cloud device, and the method includes:

[0180] S601, determining algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated;

[0181] S602, sending the algorithm startup parameters to a target camera;

[0182] S603, receiving at least one video material obtained by the target camera based on the algorithm startup parameters;

[0183] S604: Generate a video required by the user based on the at least one video material.

[0184] In some embodiments, determining algorithm startup parameters for detecting the captured video includes:

[0185] Acquire the demand information input by the user;

[0186] Performing semantic analysis on the demand information to obtain analysis results;

[0187] If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determine the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, output user prompt information (for example, the user may be prompted to re-enter, or the user may be prompted to select an alternative input scheme, and the specific prompt content is not limited in the embodiment of the present application);

[0188] If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, user prompt information is output (for example, the user may be prompted to re-enter, or the user may be prompted to select an alternative input scheme, and the specific prompt content is not limited in the embodiments of this application).

[0189] For example, see Figure 7 As shown, the method for the cloud device to determine the algorithm startup parameters includes:

[0190] S701: Obtain demand information input by the user.

[0191] S702: Perform semantic analysis on the demand information input by the user to obtain analysis results.

[0192] S703: When the analysis result includes a person, searching for a preset algorithm for detecting the person.

[0193] S704, determine whether there is a preset algorithm for detecting the person, if yes, execute step S705, otherwise, execute step S708.

[0194] S705: Determine the character behavior detection capability supported by the preset algorithm for detecting the character, and the character behavior in the analysis result.

[0195] S706: Determine whether the detection capability supports detecting the character behavior. If yes, execute step S707; otherwise, execute step S708.

[0196] S707: Determine algorithm startup parameters.

[0197] S708: Output user prompt information.

[0198] In some embodiments, for example, after semantic analysis of the demand information for input is performed through the cloud, an algorithm to be loaded (for example, a motion detection algorithm) is selected based on the subject information therein;

[0199] Before the algorithm is started, the cloud reads the behavior analysis capabilities supported by the algorithm to be started, and divides the algorithm's behavior analysis capabilities into multiple subsets. It analyzes the parsed subject behavior. If the behavior in the subject input by the user does not belong to the behavior that the algorithm can analyze, nor does it belong to the subset behavior, the user is fed back to re-enter the information and is given relevant information prompts (for example, it is prompted that the system does not support the behavior currently entered by the user, and it is recommended to select from the supported behaviors and provide alternative behaviors).

[0200] For example, the video detection algorithm supported by a home security camera can detect the following behaviors: sitting, walking, standing, lying, reading, looking at a mobile phone, cleaning, running on a treadmill, and doing yoga on a yoga mat. Then, these behaviors can be further divided into the following subsets:

[0201] Rest subset, including: sitting and lying down.

[0202] The leisure subset includes: reading and looking at mobile phones.

[0203] The exercise subset includes: running on a treadmill, doing yoga on a yoga mat, and walking.

[0204] If the system supports detection and analysis of user input behavior, the algorithm startup parameters are automatically configured according to the behavior to be detected, for example:

[0205] 1. Automatically select the algorithm to be started based on the topic entered by the user.

[0206] For example, if the user inputs the topic as sports, then it is necessary to start the detection algorithms such as running and doing yoga.

[0207] 2. Automatically configure the detection area.

[0208] For example, if the user inputs the theme of dinner party, after the algorithm detects the eating behavior, it cannot define it as a dinner party. It also needs information about the number of people. At this time, the area around the dining table will be determined as the detection area, and the number of people eating will be determined based on the area.

[0209] 3. Automatically configure detection sensitivity.

[0210] For example, 10 sensitivities can be set to automatically adjust the sensitivity according to factors such as the brightness of the environment, the distance between the target and the camera, etc. Specifically, for example, if the target is a dog, when the environment is dark, a higher detection sensitivity is set, for example, when there is a 90% probability of determining that the target is a dog, the target can be finally determined to be a dog; otherwise, when the environment is normal or bright, a lower detection sensitivity is set, for example, when there is a 95% probability of determining that the target is a dog, the target can be finally determined to be a dog.

[0211] 4. Automatically determine whether to start voice-assisted detection based on the detection scenario.

[0212] For example, for family gathering scenarios, voice-assisted detection needs to be enabled.

[0213] Voice and video algorithm detection can enrich the algorithm's detection capabilities and improve the accuracy of algorithm detection.

[0214] After the algorithm startup parameters are deployed, the cloud can send the algorithm startup parameters (including the video detection algorithm to be started, the detection area on the image, the detection sensitivity, the indication information of whether to start voice detection, etc.) to the camera.

[0215] When the camera side has finished screening the video material, the statistical data of the text information and the video material will be uploaded to the cloud and saved in the cloud database. When the time limit set by the user is reached, the cloud starts video synthesis. Before the video is synthesized, the statistical data of the text information corresponding to the video material will be sent to the text processing model to be converted into a text, and the text will be superimposed on the video. For example, the final video is synthesized by n materials, and text will be superimposed on each piece of material. The text is generated by using a natural language processing model after statistics of the key information detected by the algorithm. There may be different characters and behaviors in each piece of material. Each behavior of each character can correspond to the generation of text information to explain the character's behavior. The text information and the video are highly correlated. Therefore, the embodiment of the present application synthesizes the key text into a text, so that the synthesized video carries text information related to the video, and the text information is superimposed on the video during video synthesis, so as to realize the automatic generation of graphic video.

[0216] In some embodiments, matching background music may also be automatically selected from an audio material library according to a theme input by the user;

[0217] In some embodiments, transition effects can also be added at the joints of the materials to achieve a smooth transition of the video at the joints. The transition effects can be fade-in, fade-out, overlay transition, blocking transition, etc. Finally, the videos are merged and the background music is cut to the same length as the video.

[0218] In some embodiments, if the presence of audio material is detected, the volume of the background music is further adjusted to be lower than the volume of the audio material.

[0219] In some embodiments, if there is too much material, it can be automatically split according to the set video length to generate multiple videos.

[0220] In summary, in the embodiments of the present application, the user only needs to input the required information such as the theme and time limit of the video to be generated through text, and the entire system will automatically perform material screening, material splicing, text generation, music addition, and special effect synthesis in the video source captured by the camera in real time, completing video editing in one stop. The user only needs to export the video with one click to get the required video. In addition, the embodiments of the present application can eliminate useless video materials during video capture, and the solution of material screening during video capture does not require the preparation of a large amount of materials, saves memory, and avoids a large amount of secondary processing.

[0221] The following is an introduction to the equipment, devices, and systems provided in the embodiments of the present application, wherein explanations or examples of technical features that are the same as or corresponding to those described in the above method will not be repeated later. Example

[0222] An electronic device provided in an embodiment of the present application, see Figure 8 , for example, comprising: a processor 800 and a memory 820; wherein:

[0223] When the electronic device provided in the embodiment of the present application is used as a device on the camera side (which may be the camera itself or any other electronic device with image acquisition and processing functions):

[0224] The processor 800 is used to read the program in the memory 820 and execute the following process:

[0225] Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated;

[0226] Based on the algorithm startup parameters, start the preset algorithm to detect the collected video;

[0227] If the detection result of the current frame image is consistent with the requirement information pre-input by the user, at least one video material is determined starting from the current frame image; wherein the at least one video material is used to generate the video required by the user.

[0228] In some embodiments, starting from the current frame image, determining at least one segment of video material includes:

[0229] The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

[0230] In some embodiments, the processor 800 is further configured to read a program in the memory 820 and execute the following process:

[0231] In the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user.

[0232] In some embodiments, determining statistical data includes:

[0233] When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material;

[0234] If new video material collection is not started, then:

[0235] If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table;

[0236] If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table;

[0237] If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

[0238] In some embodiments, the processor 800 is further configured to read a program in the memory 820 and execute the following process:

[0239] Based on the at least one video material, screening the video material;

[0240] The screened video material is sent to the cloud, and the video required by the user is generated through the cloud.

[0241] In some embodiments, based on the at least one video material, performing video material screening includes:

[0242] For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

[0243] In some embodiments, based on the at least one video material, performing video material screening includes:

[0244] For video materials of multiple days, the video materials are classified according to a single day to obtain classification results;

[0245] For a single day's video material, the video material is classified according to time to obtain a classification result;

[0246] For each of the above classification results:

[0247] If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing;

[0248] If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold.

[0249] When the electronic device provided in the embodiment of the present application is used as a cloud device (which may be a server or any other electronic device with an image processing function):

[0250] The processor 800 is used to read the program in the memory 820 and execute the following process:

[0251] Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated;

[0252] Sending the algorithm startup parameters to the target camera;

[0253] Receiving at least one video material obtained by the target camera based on the algorithm startup parameters;

[0254] A video required by a user is generated based on the at least one video material.

[0255] In some embodiments, determining algorithm startup parameters for detecting the captured video includes:

[0256] Acquire the demand information input by the user;

[0257] Performing semantic analysis on the demand information to obtain analysis results;

[0258] If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determining the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, outputting user prompt information;

[0259] If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, then the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, then user prompt information is output.

[0260] The transceiver 810 is configured to receive and send data under the control of the processor 800 .

[0261] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 800 and various circuits of memory represented by memory 820 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 810 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, including wireless channels, wired channels, optical cables, and other transmission media. For different user devices, the user interface 830 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0262] The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 800 when performing operations.

[0263] In some embodiments, the processor 800 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor may also adopt a multi-core architecture.

[0264] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0265] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here. Example

[0266] See also Fig. 9 , a video generation device provided in an embodiment of the present application includes:

[0267] The algorithm startup parameter determination unit 901 is used to determine the algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the requirement information pre-entered by the user, and the requirement information is information about the video to be generated;

[0268] The video real-time detection unit 902 is used to start a preset algorithm to detect the collected video based on the algorithm startup parameters;

[0269] The video material determination unit 903 is used to determine at least one segment of video material starting from the current frame image if the detection result for the current frame image is consistent with the requirement information pre-input by the user; wherein the at least one segment of video material is used to generate the video required by the user.

[0270] In some embodiments, starting from the current frame image, determining at least one segment of video material includes:

[0271] The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

[0272] In some embodiments, the video material determination unit 903 is further configured to:

[0273] In the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user.

[0274] In some embodiments, determining statistical data includes:

[0275] When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material;

[0276] If new video material collection is not started, then:

[0277] If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table;

[0278] If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table;

[0279] If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

[0280] In some embodiments, the video material determination unit 903 is further configured to:

[0281] Based on the at least one video material, screening the video material;

[0282] The screened video material is sent to the cloud, and the video required by the user is generated through the cloud.

[0283] In some embodiments, based on the at least one video material, performing video material screening includes:

[0284] For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

[0285] In some embodiments, based on the at least one video material, performing video material screening includes:

[0286] For video materials of multiple days, the video materials are classified according to a single day to obtain classification results;

[0287] For a single day's video material, the video material is classified according to time to obtain a classification result;

[0288] For each of the above classification results:

[0289] If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing;

[0290] If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold. Example

[0291] Another video generation device provided in the embodiment of the present application is shown in FIG. Fig.10 ,include:

[0292] The algorithm startup parameter determination unit 110 is used to determine the algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the requirement information pre-entered by the user, and the requirement information is information about the video to be generated;

[0293] An algorithm startup parameter sending unit 120, used to send the algorithm startup parameters to a target camera;

[0294] The video material acquisition unit 130 is used to receive at least one video material obtained by the target camera based on the algorithm startup parameters;

[0295] The video synthesis unit 140 is used to generate a video required by a user based on the at least one video material.

[0296] In some embodiments, determining algorithm startup parameters for detecting the captured video includes:

[0297] Acquire the demand information input by the user;

[0298] Performing semantic analysis on the demand information to obtain analysis results;

[0299] If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determining the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, outputting user prompt information;

[0300] If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, then the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, then user prompt information is output.

[0301] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0302] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0303] Any of the devices or apparatuses provided in the embodiments of the present application may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. It may include a central processing unit (CPU), a memory, an input / output device, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.

[0304] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiment of the present application, the memory may be used to store the program of any of the methods provided in the embodiment of the present application.

[0305] The processor calls the program instructions stored in the memory, and the processor is used to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions. Example

[0306] See also Fig.11 , a video generation system provided by an embodiment of the present application includes: a cloud device 111 and at least one camera 112;

[0307] The cloud device 111 is used to determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on the demand information pre-entered by the user, and the demand information is information about the video to be generated; the algorithm startup parameters are sent to the target camera; at least one video material obtained by the target camera based on the algorithm startup parameters is received; based on the at least one video material, the video required by the user is generated;

[0308] The camera 112 is used to receive algorithm startup parameters sent by the cloud device for detecting the collected video; based on the algorithm startup parameters, start a preset algorithm to detect the collected video; if the detection result of the current frame image is consistent with the requirement information pre-entered by the user, then starting from the current frame image, determine at least one video material; and send the at least one video material to the cloud device.

[0309] In some embodiments, the cloud device 111 determines algorithm startup parameters for detecting the collected video, including:

[0310] Acquire the demand information input by the user;

[0311] Performing semantic analysis on the demand information to obtain analysis results;

[0312] If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determining the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, outputting user prompt information;

[0313] If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, then the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, then user prompt information is output.

[0314] In some embodiments, the camera 112 determines at least one segment of video material starting from the current frame image, including:

[0315] The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

[0316] In some embodiments, the camera 112 is also used to:

[0317] In the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user.

[0318] In some embodiments, determining statistical data includes:

[0319] When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material;

[0320] If new video material collection is not started, then:

[0321] If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table;

[0322] If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table;

[0323] If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

[0324] In some embodiments, the camera 112 is also used to:

[0325] Based on the at least one video material, screening the video material;

[0326] The screened video material is sent to the cloud, and the video required by the user is generated through the cloud.

[0327] In some embodiments, based on the at least one video material, performing video material screening includes:

[0328] For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

[0329] In some embodiments, based on the at least one video material, performing video material screening includes:

[0330] For video materials of multiple days, the video materials are classified according to a single day to obtain classification results;

[0331] For a single day's video material, the video material is classified according to time to obtain a classification result;

[0332] For each of the above classification results:

[0333] If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing;

[0334] If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold.

[0335] The present application embodiment also provides a computer program product or computer program, which includes a computer instruction, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction so that the computer device performs any of the methods described in the above embodiments. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0336] The embodiment of the present application provides a computer-readable storage medium for storing computer program instructions used by the apparatus provided in the embodiment of the present application, which includes a program for executing any of the methods provided in the embodiment of the present application. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0337] The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0338] It should be understood that:

[0339] The access technology through which entities in the communication network transmit traffic can be any suitable current or future technology, such as WLAN (Wireless Local Access Network), WiMAX (Worldwide Interoperability for Microwave Access), LTE, LTE-A, 5G, Bluetooth, infrared, etc.; in addition, the embodiments can also apply wired technology, for example, IP-based access technology, such as a wired network or a fixed line.

[0340] Embodiments suitable for being implemented as software code or a portion thereof and running using a processor or processing functionality are independent of the software code and may be specified using any known or future developed programming language, such as a high-level programming language such as objective-C, C, C++, C#, Java, Python, Javascript, other scripting languages, etc., or a low-level programming language such as machine language or assembler.

[0341] The implementation of the embodiments is hardware independent and may be implemented using any known or future developed hardware technology or any mixture thereof, such as a microprocessor or CPU (central processing unit), MOS (metal oxide semiconductor), CMOS (complementary MOS), BiMOS (bipolar MOS), BiCMOS (bipolar CMOS), ECL (emitter coupled logic) and / or TTL (transistor-transistor logic).

[0342] Embodiments may be implemented as separate devices, apparatuses, units, components, or functions, or in a distributed manner, for example, one or more processors or processing functions may be used or shared in a process, or one or more processing segments or processing portions may be used and shared in a process, wherein one physical processor or more than one physical processor may be used to implement one or more processing portions dedicated to a specific process as described.

[0343] The device may be implemented by a semiconductor chip, a chipset, or a (hardware) module including such a chip or chipset.

[0344] The embodiments may also be implemented as any combination of hardware and software, such as ASIC (Application Specific IC (Integrated Circuit)) components, FPGA (Field Programmable Gate Array) or CPLD (Complex Programmable Logic Device) components or DSP (Digital Signal Processor) components.

[0345] The embodiments may also be implemented as a computer program product including a computer usable medium having computer readable program code embodied therein, the computer readable program code being adapted to perform the processes as described in the embodiments, wherein the computer usable medium may be a non-transitory medium.

[0346] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0347] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0348] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0349] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0350] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video generation method, characterized in that: The method comprises: Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated; Based on the algorithm startup parameters, start the preset algorithm to detect the collected video; If the detection result of the current frame image is consistent with the requirement information pre-entered by the user, then starting from the current frame image, at least one video material is determined; wherein the at least one video material is used to generate the video required by the user; in the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user, wherein the statistical data includes: characters, behaviors, and frequency, duration, and coordinates of the behaviors; The statistical data of the at least one video material and the corresponding text information are sent to the cloud, and the video required by the user is generated by the cloud, wherein the cloud sends the statistical data of the text information corresponding to the video material to the text processing model to convert it into text, and superimposes the text on the video.

2. The method according to claim 1, characterized in that Starting from the current frame image, at least one video material is determined, including: The timestamp of the current frame image is used as the initial time value of the comparison frame, and starting from the current frame image, for each frame image, based on the time difference between the timestamp of the frame image and the comparison frame, it is determined whether the acquisition of the current segment of video material is completed, wherein when the time difference is less than a preset threshold, the frame image is counted into the current segment of video material, and the time value of the comparison frame is updated to the timestamp of the frame image; when the time difference is greater than or equal to the preset threshold, it is determined that the acquisition of the current segment of video material is completed, and the acquisition of new video material is started, the frame image is counted into the new video material, and the time value of the comparison frame is updated to the timestamp of the frame image.

3. The method according to claim 1, characterized in that The determined statistical data include: When starting to collect new video material, record the behavior, starting frame identifier, and coordinate information of the character in the first frame image of the new video material, and record the behavior frequency of the character according to the maintained global character behavior statistical frequency table; wherein the global character behavior statistical frequency table is used to count the frequency of each behavior of each character in the at least one video material; If new video material collection is not started, then: If a new behavior appears in the current frame image, the start frame identifier of the new behavior, the end frame identifier of the previous behavior, and the new behavior frequency of the character are recorded according to the maintained global character behavior statistical frequency table; If no new behavior appears in the current frame image, but a new person appears, then the new person, the behavior of the new person, the start frame identifier, the coordinate information, and the behavior frequency of the new person are recorded according to the maintained global person behavior statistical frequency table; If no new behavior or new person appears in the current frame image, but a person disappears, the disappeared person is determined, and the end frame identifier of the disappeared person is recorded.

4. The method according to claim 1, characterized in that: The method further comprises: Based on the at least one video material, screening the video material; Sending statistical data of the at least one video material and corresponding text information to the cloud includes: The statistical data of the filtered video materials and the corresponding text information are sent to the cloud.

5. The method according to claim 4, characterized in that Based on the at least one video material, video material screening is performed, including: For every two video materials, the two video materials are compared. If the characters and / or character behaviors of the two video materials are different, the two video materials are retained; otherwise, the video material with higher video quality is retained.

6. The method according to claim 4, characterized in that Based on the at least one video material, video material screening is performed, including: For video materials of multiple days, the video materials are classified according to a single day to obtain classification results; For a single day's video material, the video material is classified according to time to obtain a classification result; For each of the above classification results: If the character behaviors in the classification result are the same behavior, and the duration of the same behavior exceeds a preset threshold, the classification result is subjected to frame extraction processing; If the character behaviors in the classification result include different behaviors, the classification result is sliced ​​based on the behaviors of each character in the classification result to obtain multiple slices, each slice corresponding to one behavior; and frame extraction is performed on the slices whose behavior duration exceeds a preset threshold.

7. A video generation method, characterized in that: The method comprises: Determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated; The algorithm startup parameters are sent to a target camera, so that the target camera: based on the algorithm startup parameters, starts a preset algorithm to detect the collected video; if the detection result of the current frame image is consistent with the required information pre-entered by the user, then at least one video material is determined starting from the current frame image; and in the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user, wherein the statistical data includes: characters, behaviors, and the frequency, duration, and coordinates of the behaviors; the statistical data of the at least one video material and the corresponding text information are sent to the cloud; Receiving statistical data of at least one video material and corresponding text information obtained by the target camera based on the algorithm startup parameters; Based on the statistical data of at least one video material and the corresponding text information, a video required by a user is generated, which includes sending the statistical data of the text information corresponding to the video material into a text processing model to convert it into text, and superimposing the text on the video.

8. The method according to claim 7, characterized in that The step of determining the algorithm startup parameters for detecting the collected video includes: Acquire the demand information input by the user; Performing semantic analysis on the demand information to obtain analysis results; If the analysis result includes a person, and when there is a preset algorithm for detecting the person, determining the person behavior detection capability supported by the preset algorithm; if there is no preset algorithm for detecting the person, outputting user prompt information; If the analysis result also includes character behavior, and the character behavior detection capability supported by the preset algorithm supports detecting the character behavior, then the algorithm startup parameters are determined; if the character behavior detection capability supported by the preset algorithm does not support detecting the character behavior, then user prompt information is output.

9. A video generating device, characterized in that: The device comprises: An algorithm startup parameter determination unit, used to determine an algorithm startup parameter for detecting the collected video; wherein the algorithm startup parameter is determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated; A video real-time detection unit, used to start a preset algorithm to detect the collected video based on the algorithm startup parameters; A video material determination unit is used to determine at least one video material starting from the current frame image if the detection result for the current frame image is consistent with the demand information pre-input by the user; wherein the at least one video material is used to generate the video required by the user, and in the process of determining the at least one video material, determine statistical data, and the statistical data is used to generate text information in the video required by the user, wherein the statistical data includes: characters, behaviors, and frequency, duration, and coordinates of the behaviors; the statistical data of the at least one video material and the corresponding text information are sent to the cloud, and the video required by the user is generated through the cloud, wherein the cloud sends the statistical data of the text information corresponding to the video material to the text processing model to convert it into text, and superimposes the text on the video.

10. A video generating device, characterized in that: The device comprises: An algorithm startup parameter determination unit, used to determine an algorithm startup parameter for detecting the collected video; wherein the algorithm startup parameter is determined based on demand information pre-entered by a user, and the demand information is information about the video to be generated; An algorithm startup parameter sending unit is used to send the algorithm startup parameters to a target camera, so that the target camera: based on the algorithm startup parameters, starts a preset algorithm to detect the collected video; if the detection result of the current frame image is consistent with the demand information pre-entered by the user, then starting from the current frame image, at least one video material is determined; and in the process of determining the at least one video material, statistical data is determined, and the statistical data is used to generate text information in the video required by the user, wherein the statistical data includes: characters, behaviors, and frequency, duration, and coordinates of the behaviors; and the statistical data of the at least one video material and the corresponding text information are sent to the cloud; A video material acquisition unit, configured to receive statistical data of at least one video material and corresponding text information obtained by the target camera based on the algorithm startup parameters; The video synthesis unit is used to generate the video required by the user based on the statistical data of the at least one video material and the corresponding text information, including sending the statistical data of the text information corresponding to the video material into the text processing model to convert it into text, and superimposing the text on the video.

11. A video generation system, characterized in that: The system includes: a cloud device and a camera; The cloud device is used to determine algorithm startup parameters for detecting the collected video; wherein the algorithm startup parameters are determined based on demand information pre-entered by the user, and the demand information is information about the video to be generated; the algorithm startup parameters are sent to the camera; statistical data of at least one video material and corresponding text information obtained by the camera based on the algorithm startup parameters are received; based on the statistical data of the at least one video material and the corresponding text information, the video required by the user is generated, including sending the statistical data of the text information corresponding to the video material into a text processing model to convert it into text, and superimposing the text on the video; The camera is used to receive algorithm startup parameters sent by the cloud device for detecting the collected video; based on the algorithm startup parameters, start a preset algorithm to detect the collected video; if the detection result of the current frame image is consistent with the demand information pre-entered by the user, then start from the current frame image to determine at least one video material; in the process of determining the at least one video material, determine statistical data, and the statistical data is used to generate text information in the video required by the user, wherein the statistical data includes: characters, behaviors, and frequency, duration, and coordinates of the behaviors; and send the statistical data of the at least one video material and the corresponding text information to the cloud device.

12. An electronic device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 8 according to the obtained program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute the method according to any one of claims 1 to 8.

Images