Image generation method, device, equipment and computer storage medium

Abstract

The present invention discloses an image generation method, device, equipment and computer storage medium, the method comprising: when an event trigger is detected, video frame images are collected as a first cover image set within the duration of the event, object data and screen change data corresponding to each first cover image in the first cover image set are obtained, the priority of the first cover image is confirmed based on the object data and the screen change data, the second cover image of the event is confirmed based on the priority of each first cover image, and the second cover image is sent to the client. The camera obtains the first cover image set by collecting video frame images, and the second cover image is obtained by screening from the first cover image set, and the second cover image is sent to the client, and the client further classifies based on the second cover image after the camera screens, thereby reducing the cover classification cost of the client and improving the cover classification efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image generation method, device, equipment and a computer storage medium. Background Art

[0002] With the development of technology, many digital capture devices can capture videos as well as static images. When faced with a large number of videos, users may prefer to be able to obtain the content of the video without having to view the entire video. Therefore, the role of video cover in video analysis and video retrieval is becoming more and more important.

[0003] However, the image generation methods in the prior art cannot completely and accurately generate satisfactory video cover images. Summary of the invention

[0004] The main purpose of the present invention is to provide an image generation method, device, equipment and computer storage medium, aiming to accurately and efficiently generate a cover image. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides an image generation method, which is applied to a camera, comprising:

[0006] When an event trigger is detected, collecting video frame images within the duration of the event as a first cover image set;

[0007] Acquire object data and screen change data corresponding to each first cover image in the first cover image set;

[0008] determining a priority of the first cover image based on the object data and the screen change data;

[0009] A second cover image of the event is confirmed based on the priority of each of the first cover images, and the second cover image is sent to the client. The second cover image is used by the client to generate a second cover image set based on the second cover image, and the second cover image set is classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0010] In a second aspect, an embodiment of the present application provides an image generation method, which is applied to a client, comprising:

[0011] Acquire a second cover image of each event captured by the camera, and generate a second cover image set, wherein the second cover image is obtained by the camera based on object data and screen change data corresponding to each first cover image in the first cover image set corresponding to the event;

[0012] The second cover images in the second cover image set are classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0013] In a third aspect, an embodiment of the present application provides an image generating device, including:

[0014] An image acquisition module, configured to acquire video frame images as a first cover image set within the duration of the event when an event trigger is detected;

[0015] A data acquisition module, used to acquire object data and screen change data corresponding to each first cover image in the first cover image set;

[0016] A priority confirmation module, used for confirming the priority of the first cover image based on the object data and the screen change data;

[0017] A cover confirmation module is used to confirm the second cover image of the event based on the priority of each of the first cover images, and send the second cover image to the client. The second cover image is used by the client to generate a second cover image set based on the second cover image, and the second cover image set is classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0018] In a fourth aspect, an embodiment of the present application provides an image generating device, including:

[0019] A cover acquisition module, used to acquire a second cover image of each event captured by the camera, and generate a second cover image set, wherein the second cover image is obtained by the camera based on object data and screen change data corresponding to each first cover image in the first cover image set corresponding to the event;

[0020] The cover classification module is used to classify each of the second cover images in the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0021] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above method when executed by the processor.

[0022] In a sixth aspect, an embodiment of the present application provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0023] In an embodiment of the present application, when a camera detects an event trigger, it collects video frame images as a first cover image set within the duration of the event, obtains object data and screen change data corresponding to each first cover image in the first cover image set, confirms the priority of the first cover image based on the object data and screen change data, confirms the second cover image of the event based on the priority of each first cover image, and sends the second cover image to the client. The client generates a second cover image set based on the second cover image corresponding to each event, and classifies the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene. During the detection process of the camera, the collected video frame images are roughly screened to obtain the cover image corresponding to the triggering event, so that the client only needs to classify the cover image to obtain the cover image in different scenes, which reduces the client's computing burden, and can help users quickly locate the video information of interest by classifying the cover, thereby improving the efficiency of video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is an example schematic diagram of an image generation method provided in an embodiment of the present application;

[0026] Figure 2 It is a flowchart of an image generation method provided in an embodiment of the present application;

[0027] Figure 3 It is a flowchart of an image generation method provided in an embodiment of the present application;

[0028] Figure 4 is a flow chart of a method for generating an image provided by an embodiment of the present application;

[0029] Figure 5 is a flow chart of a method for generating an image provided by an embodiment of the present application;

[0030] Figure 6 It is a flowchart of an image generation method provided in an embodiment of the present application;

[0031] Figure 7 is a flow chart of a method for generating an image provided by an embodiment of the present application;

[0032] Figure 8is a flow chart of a method for generating an image provided by an embodiment of the present application;

[0033] Fig. 9 is an example schematic diagram of an image generation method provided in an embodiment of the present application;

[0034] Fig.10 is a structural schematic diagram of an image generating device provided in an embodiment of the present application;

[0035] Fig.11 is a structural schematic diagram of an image generating device provided in an embodiment of the present application;

[0036] Fig.12 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] The image generating device can be a camera or a module in the camera for implementing the image generating method. When an event trigger is detected, the image generating device can collect video frame images as a first cover image set within the duration of the event, obtain object data and screen change data corresponding to each first cover image in the first cover image set, confirm the priority of the first cover image based on the object data and the screen change data, confirm the second cover image of the event based on the priority of each first cover image, and send the second cover image to the client.

[0039] Furthermore, the image generating device can be a client or a module in the client for implementing the image generating method. The image generating device can obtain the second cover image of each event captured by the camera and generate a second cover image set. The second cover image is obtained by the camera based on the object data and screen change data corresponding to each first cover image in the first cover image set corresponding to the event. The second cover images in the second cover image set are classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0040] Please also see Figure 1, an example schematic diagram of an image generation method is provided for an embodiment of the present application, the camera can collect video frame images within an event duration event as a first cover image set, by obtaining object data and screen change data corresponding to each first cover image in the first cover image set, confirming the priority of the first cover image based on the screen change data and the object data, confirming the second cover image from the first cover image set according to the priority, the camera sends the second cover image corresponding to the event to the terminal device, the terminal device generates a second cover set after receiving the second cover images corresponding to each event, and classifies each second cover image in the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0041] The image generation method provided by the present application is described in detail below in conjunction with specific embodiments.

[0042] See also Figure 2 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S101-S104.

[0043] S101, when an event trigger is detected, collecting video frame images within the duration of the event as a first cover image set;

[0044] S102, obtaining object data and screen change data corresponding to each first cover image in the first cover image set;

[0045] S103, confirming the priority of the first cover image based on the object data and the screen change data;

[0046] S104, confirming the second cover image of the event based on the priority of each of the first cover images, sending the second cover image to the client, the second cover image is used by the client to generate a second cover image set based on the second cover image, and classifying the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0047] The video processing method provided in the embodiment of the present application can be used in camera recording and monitoring scenarios. In the camera recording scenario, the length of the recorded video may be as long as more than ten hours. When the user watches the recorded video playback, it is difficult to find the information of interest or value from it. It takes a lot of time to watch the video content and find the content of interest, which is time-consuming and inefficient. Therefore, the image generation method of the present application is proposed, which automatically generates a cover image through a camera, so that it is convenient to understand the important information in the video when the video content is not viewed. It should be noted that the cover image can also be called a summary image. By viewing the cover image, the user can roughly understand the content of a video, thereby screening the videos that have not been fully viewed and finding the video content of interest.

[0048] The following is a detailed description of each step:

[0049] S101, when an event trigger is detected, collecting video frame images within the duration of the event as a first cover image set;

[0050] Specifically, an event refers to a video event obtained by the camera based on the recognition function of the preset focus content in the video, such as screen changes, people appearing, people leaving, pets appearing, vehicles appearing, moving, etc. For example, a human event can be identified in the following way: the camera continuously collects images, and the camera's built-in artificial intelligence (AI) algorithm detects whether there is a face or human figure in the image. If the AI ​​algorithm detects a face or human figure, it reports a human event. The specific detection methods for other events are not detailed here.

[0051] In a feasible implementation, the camera will record when an event is triggered and save the event video. Therefore, when an event is detected, video frame images are collected during the recording process, that is, video frame images are collected during the duration of the event, and each collected video frame image is used as the first cover image, and a first cover image set is generated based on the collected multiple first cover images. The first cover image set may include one or more first cover images, preferably multiple first cover images. Optionally, video frame images may be collected as the first cover image set within the duration of the event based on a preset collection interval. Exemplarily, the video frame image collected when the event is triggered is used as the first first cover image of the event, and then a first cover image is collected every 3 seconds or every 5 seconds until the camera detects that the current event ends, or a new event starts.

[0052] In a feasible implementation, the camera can also record when no event occurs, such as recording within a set recording duration, and based on a preset acquisition interval, collect video frames within the recording duration as a third cover image set, obtain object data and screen change data corresponding to each third cover image in the third cover image set; confirm the priority of the third cover image based on the object data and screen change data; confirm the second cover image of the event based on the priority of each first cover image, and send the second cover image to the client. It can be understood that the camera has an event recognition function, and the occurrence of an event is usually the content that the user is concerned about in the recording. Therefore, only obtaining the cover image corresponding to the event when the event is triggered can increase the probability of the camera obtaining useful information, reduce the waste of unnecessary computing power, and generate a cover image that better meets user needs.

[0053] S102, obtaining object data and screen change data corresponding to each first cover image in the first cover image set;

[0054] Specifically, after the camera captures the first cover images, it can detect the first cover images to obtain object data and screen change data corresponding to each first cover image.

[0055] Specifically, the object may be an object of interest set in the video screen. For different camera application scenarios, different object of interest recognition capabilities may be configured for the camera, thereby obtaining object data in the video. Objects may include vehicles, people, and pets, etc. Preferably, object data refers to movable objects appearing in the video screen (image) and data obtained by further analyzing the object, such as the number of objects, the size of the object image, the age of the object, the position of the object, the part of the object, etc. It is understandable that objects may be divided into known objects and unknown objects based on pre-stored object information. Exemplarily, when the object is a person, by identifying the image captured during the recording process, if the object in the image matches the pre-stored object information, it may be a known object, such as a family member of the user. If it does not match the pre-stored object information, it may be a stranger, such as a courier.

[0056] Specifically, the picture change data refers to the identified video picture change data, such as the picture change area ratio, the newly added content of the picture, the reduced content of the picture, etc. By comparing the currently captured video frame image with the previously or later captured video frame image, the same and different parts in the picture can be obtained, thereby obtaining the picture change data.

[0057] S103, confirming the priority of the first cover image based on the object data and the screen change data;

[0058] It can be understood that the object data and screen change data of the first cover image can be used to characterize the information richness of the cover image. In the process of selecting the cover image, the first cover image with rich information or the one that the user is interested in is preferred. Therefore, the priority of the first cover image can be confirmed based on the object data and the screen change data. Specifically, the priority of the first cover image is confirmed based on the data already contained in the object data and the data obtained by further analyzing the object data. Exemplarily, whether the object is contained is confirmed based on the object data, and the priority of the first cover image containing the object is higher than the first cover image not containing the object; if the object is contained, the priority, number of objects, and completeness of the object display can be confirmed. The higher the object priority, the higher the priority of the first cover image, the more objects, the higher the priority of the first cover image, and the higher the completeness of the object display, the higher the priority of the first cover image. If the object is not contained, the priority can be confirmed based on the screen change data, for example, the proportion of the screen change area of ​​the first cover image is confirmed based on the screen change data, and the higher the proportion of the change area, the higher the priority of the first cover image. In addition, when the object data indicates that there is an object in the first cover image, the priority of the first cover image can also be confirmed in combination with the screen change data.

[0059] S104, confirming the second cover image of the event based on the priority of each of the first cover images, sending the second cover image to the client, the second cover image is used by the client to generate a second cover image set based on the second cover image, and classifying the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0060] Specifically, the priorities of the first cover images are compared, and the first cover image with the highest priority is selected, or a plurality of first cover images with high priorities are selected as the second cover image.

[0061] In a feasible implementation, the method for comparing the priority of the first cover image can be: each time a first cover image is obtained, the object data and screen change data of the first cover image are obtained, the priority of the first cover image is confirmed, the priority of the obtained first cover image is compared with the priority of the first cover image obtained before the current first cover image, the image with the higher priority is confirmed as the cover image to be selected, new first cover images are continuously obtained and the priorities are compared to update the cover images to be selected until the event ends, and the latest obtained cover image to be selected is used as the second cover image.

[0062] In addition, based on the object data and the screen change data, confirming the priority of the first cover image may also include: in the process of real-time recording or real-time monitoring by the camera, by setting a preset time interval, regularly screening the cover, that is, comparing the priority of the first cover image obtained based on the preset time interval, and confirming the cover image to be selected. The preset time interval can be set according to actual needs. For example, when recording, select an optimal cover every 1 minute / 5 minutes, and the cover corresponds to the recording time point.

[0063] In an embodiment of the present application, when an event trigger is detected, video frame images are collected as a first cover image set within the duration of the event, object data and screen change data corresponding to each first cover image in the first cover image set are obtained, and the priority of the first cover image is confirmed based on the object data and the screen change data. The second cover image of the event is confirmed based on the priority of each first cover image, and the second cover image is sent to the client. The camera obtains the first cover image set by collecting video frame images, and obtains the second cover image from the first cover image set by screening, and sends the second cover image to the client. The client further classifies the second cover image based on the second cover image screened by the camera, thereby reducing the cover classification cost of the client and improving the cover classification efficiency. Based on the classified scene cover image set, the problem of efficient playback of recorded videos can be solved at a low cost.

[0064] See also Figure 3 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include the following steps S201-S203.

[0065] S201, based on the object data, confirming whether there is a target object in the first cover image;

[0066] Specifically, by analyzing the object data, it can be confirmed whether the target object exists in the first cover image. The target object is a pre-selected object of interest. It is understandable that the object data obtained by recognizing the first cover image may contain one or more objects, and currently only the target object needs to be paid attention to. However, the one or more objects may not contain the target object that needs to be detected in the current event. Therefore, it is confirmed whether the target object exists in the first cover image.

[0067] S202: If yes, confirming the priority of the first cover image based on the target object data corresponding to the target object in the object data;

[0068] Specifically, if there is a target object in the first cover image, the target object data corresponding to the target object is confirmed in the object data, and the priority of the first cover image is confirmed based on the target object data. For example, if the object data includes vehicle data and human shape data, and the target object is a human shape, the human shape data is obtained as the target object data. The priority is confirmed based on the size of the area occupied by the target object in the target object data in the first cover image, and the priority of the first cover image with a larger human shape can be set higher.

[0069] S203: If not, confirm the priority of the first cover image based on the screen change data.

[0070] Specifically, if the target object does not exist in the object data, the priority of the first cover image can be determined according to the screen change data. For example, the screen change area ratio of the first cover image can be determined according to the screen change data, and the higher the change area ratio, the higher the priority of the first cover image.

[0071] See also Figure 4 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 4 As shown, the method of the embodiment of the present application may include the following steps S301-S302.

[0072] S301, acquiring object evaluation data of the first cover image based on the target object data;

[0073] Specifically, for the confirmed target object data, the priority can be confirmed according to the object evaluation data in the target object data. The object evaluation data is used to evaluate the recognition accuracy or information completeness of the target object. The object evaluation data can be the clarity, feature completeness, recognition probability, etc. of the target object. For example, if the current event is a pet event and the target object is a pet, the object evaluation data can be the probability of including a pet in the first cover image, or the clarity of the pet image in the first cover image.

[0074] S302: confirming the priority of the first cover image based on the object evaluation data.

[0075] Specifically, based on the object evaluation data of the first cover image obtained, the priority corresponding to the object evaluation data can be generated by a priority confirmation algorithm, or a priority score can be generated by a pre-set association relationship between the object evaluation data and the score, and then the priority score is used as the priority, without specific limitation. For example, the object evaluation data is the clarity of the pet image, and corresponding scores can be set for different clarity, the clearer the image, the higher the score, and the blurrier the image, the lower the score.

[0076] See also Figure 5, which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 5 As shown, the method in the embodiment of the present application may include the following steps S401-S402.

[0077] S401, when the target object data contains set characteristic data, obtaining an object image quality score of the first cover image;

[0078] In a feasible embodiment, the object evaluation data is the object image quality score or the object recognition confidence. By confirming whether the target object data has the set feature data, the object evaluation data to be obtained is confirmed. The set feature data is the feature that needs to be paid special attention to in the cover image, such as a human face, the front face of a car, the front face of a pet, a hand, etc. It can be understood that the front face of the car can be the side of the front windshield of the car. Since the front face image of the car can usually obtain the license plate information, annual inspection information, etc., and has more information about the vehicle, the front face of the car can be set as the set feature of interest.

[0079] Specifically, the AI ​​algorithm in the camera can be used to perform image quality scoring on the set feature data. The set feature data can be set by technical personnel according to actual conditions, and the corresponding recognition algorithm can be trained to perform quality scoring.

[0080] For example, if the feature data is set to be a human face, since the camera monitoring and collection process may collect low-quality images such as blurred faces, incorrect head angles, and images blocked by other objects, the quality indicators can be defined according to the human visual system, and the face image quality score can be generated by the face image quality analysis algorithm. The priority of the first cover image can be confirmed based on the face image quality score.

[0081] Exemplarily, assuming that the target object is a person and the feature data is set to a human face, after obtaining the first cover image, first confirm whether there is a person in the first cover image. If so, then confirm whether there is a face. If there is a face, obtain the face image quality score and confirm the priority based on the face image quality score.

[0082] S402: When the target object data does not contain set feature data, obtain the object recognition confidence of the first cover image.

[0083] Specifically, when the target object data does not contain the set feature data, the object recognition confidence of the target object in the first cover image can be obtained, and the priority of the first cover image can be determined based on the object recognition confidence. Generally speaking, the higher the object recognition confidence, the higher the possibility that the target object exists in the first cover image, and the higher the priority.

[0084] For example, assuming that the pet does not belong to the set feature data, but the pet can be the target object, when the pet is detected in the first cover image, the object recognition confidence of the pet is obtained, and the priority is confirmed according to the object recognition confidence. It can be understood that if the pet confidence of the first cover image of the current frame is 0.7, and the pet confidence of the first cover image of the next frame is 0.8, the frame image with a higher pet confidence has a higher priority.

[0085] See also Figure 6 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 6 As shown, the method of the embodiment of the present application may include the following steps S501-S502.

[0086] S501, when there are multiple target objects in the first cover image, confirming the priority of the target objects;

[0087] S502: Based on the object data, confirm the object evaluation data of the target object with the highest priority in the first cover image.

[0088] It is understandable that there may be multiple target objects. When there are multiple target objects in the first cover image, the priority of the target objects can be confirmed. Among them, the priority can be pre-set for each target object, and then the target object with the highest priority can be found according to the priorities corresponding to the multiple identified target objects. For example, the priority of people is greater than the priority of pets. Then, when there are people and pets in the first cover image, the object evaluation data of people is obtained in the object data first, and then the priority of the first cover image is determined according to the object evaluation data of people.

[0089] See also Figure 7 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 7 As shown, the method in the embodiment of the present application may include the following steps S601-S602.

[0090] S601, confirming the change area ratios of the first cover image and the historical moment image based on the screen change data, wherein the historical moment image is an image at a moment before the first cover image is captured;

[0091] S602: Determine the priority of the first cover image based on the changed area ratio.

[0092] Specifically, the first cover image is compared with the historical moment image to confirm the percentage of the changed area. The historical moment image is the image of the previous moment before the first cover image is captured. The previous moment can be a moment close to the first cover image capture moment, such as the previous frame, or it can be a gap of multiple frames. The time interval between the capture moment and the previous moment is not limited. It can be understood that by comparing the images, different parts of the image, that is, the changed area, can be obtained, and then the percentage of the changed area to the total screen area can be confirmed to obtain the percentage of the changed area.

[0093] It can be understood that if the proportion of the changed area is larger, it means that there are more changes in the image at the target moment compared to the previous moment, and the value is higher. Therefore, the priority of the first cover image can be determined based on the proportion of the changed area.

[0094] In the embodiment of the present application, by confirming whether there is a target object in the first cover image, when there is a target object in the first cover image, the target object data is obtained in the object data, and the priority of the first cover image is confirmed according to the target object data. When there is no target object in the first cover image, the priority of the first cover image is confirmed according to the screen change data, so that a cover image that is consistent with the video content and accurately summarizes the event can be efficiently generated to help users quickly review. Further, when confirming the priority of the first cover image based on the target object data, the object evaluation data of the first cover image is obtained according to the target object data, and the priority of the first cover image is confirmed based on the object evaluation data. By confirming the target object, it is only necessary to generate the priority according to the target object data, which can reduce the calculation amount of confirming the priority and improve the calculation speed; in addition, when there are multiple target objects, the target object data with the highest priority can be confirmed in the object data according to the priority of the target object, and the object evaluation data can be confirmed according to the target object data with the highest priority, so that the obtained cover image can include more target object information that users are concerned about. Furthermore, by confirming the proportion of the changed areas of the first cover image and the historical moment image based on the screen change data, and confirming the priority of the first cover image based on the proportion of the changed areas, images with larger screen changes in the first cover image can be identified, thereby improving the accuracy of outputting the second cover image.

[0095] See also Figure 8 , which is a flow chart of an image generation method provided in an embodiment of the present application. Figure 8 As shown, the method of the embodiment of the present application may include the following steps S701-S705.

[0096] S701, acquiring a second cover image of each event captured by a camera, and generating a second cover image set, wherein the second cover image is obtained by the camera based on object data and screen change data corresponding to each first cover image in a first cover image set corresponding to the event;

[0097] In a feasible implementation, the camera supports at least the following three recording modes: real-time recording, cloud recording, and card recording, that is, the video recorded by the camera can be transmitted to the client in real time for viewing, or can be transmitted to the cloud for storage, or can be stored on the camera's memory card. Correspondingly, the client can obtain the second cover image uploaded by the camera from the cloud, or receive the second cover image transmitted to the client by the camera in real time, or obtain the second cover image from the camera's memory card. The second cover image can be stored in the same location as the recorded video or stored separately.

[0098] The process of classifying the second cover image generated by the camera can be implemented on the camera, on the cloud, or on the client. In the embodiment of the present application, the client is used for implementation, and the classification processing is implemented by installing application software (Application, APP) on the client, which does not waste cloud resources, and more abundant classification processing can be achieved by updating the detection and recognition capabilities of the APP.

[0099] Specifically, the second cover image generated by the camera for each event within a preset time may be obtained, for example, the second cover image generated on the current day or the second cover image generated in the past few days, to obtain a second cover image set.

[0100] S702: Classify each of the second cover images in the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0101] Specifically, the second cover image set is classified according to the scene, and a scene cover image set corresponding to each scene is generated. Among them, the scene may include a multi-person scene, a dinner scene, a TV watching scene, a mobile phone playing scene, a homework scene, a baby, a pet, a smiling face, a human figure, a vehicle, etc. Optionally, the above-mentioned scenes can be further subdivided and classified, for example, human figure recognition can also include clothing color recognition, and human figures are classified according to color, etc. Through the recognition algorithm, the scene corresponding to the second cover image can be identified. It can be understood that since the second cover image is an image corresponding to an event, the camera already has a certain recognition and detection capability. For example, it can be recognized that the second cover image is an image with a face. Then, according to the event type corresponding to the second cover image, the image can be classified into the face category. However, whether the face is a smiling face needs to be further classified and detected by the client. Therefore, classifying the second cover image set through the recognition algorithm of the client can obtain a more detailed cover classification result.

[0102] Furthermore, in one embodiment, the method further includes:

[0103] S801, acquiring, from the camera, a target event video associated with each scene cover image in the scene cover image set;

[0104] Specifically, after the scene cover image set is obtained through classification, the second cover image in the scene cover image set is called the scene cover image, and the target event video associated with the scene cover image is obtained. The target event video refers to the event video to which the scene cover image is acquired at the corresponding time point. It is understandable that when the camera acquires the cover image corresponding to the event, the event video can also be saved accordingly, so an association relationship can be established between the scene cover image and the event video.

[0105] S802, generating a video display control based on the scene cover image and the target event video, wherein the video display control is used to receive a scene cover image selection instruction from a user to play the target event video corresponding to the scene cover image.

[0106] Specifically, after the target event video is acquired, a video display control can be generated based on the scene cover image and the target event video. When the user triggers the scene cover image selection instruction, the target event video corresponding to the scene cover image can be played. Specifically, the scene cover image can be displayed on the client in the form of an album, that is, the user can browse the scene cover image collection corresponding to each scene. Optionally, when the user clicks on the scene cover image collection, each scene cover image in the scene cover image collection will be displayed. A video display control can be set on the scene cover image. When the user clicks the virtual play button of the video display control, the video display control plays the corresponding target event video. Please refer to Fig. 9 , Fig. 9 This is a schematic diagram of an example of an image generation method provided in an embodiment of the present application. Fig. 9 The interface diagram of the client APP for classifying cover images is shown in the figure. When the user can use the intelligent retrieval function to obtain the second cover image set from the camera in real time, the cover image set is obtained after classification. Fig. 9 The intelligent search results shown on the right can be classified by scene, and can also be classified by person. For example, the images corresponding to the identified family members can be displayed. Furthermore, video display controls can be displayed, such as Fig. 9 Under the baby category, there can be multiple cover images containing babies, corresponding event videos and time points. Clicking the video display control can play the corresponding event video.

[0107] In an embodiment of the present application, a second cover image set is generated by acquiring the second cover image of each event captured by a camera, and each second cover image in the second cover image set is classified based on the scene to obtain a scene cover image set corresponding to each scene, and a target event video associated with each scene cover image in the scene cover image set is acquired from the camera, and a video display control is generated based on the scene cover image and the target event video. Videos can be quickly found based on the classified cover images, thereby satisfying the user's need to quickly view videos and conveniently viewing the original video for complete viewing.

[0108] The following will be combined with the attached Fig.10 and 11 , the image generation device provided in the embodiment of the present application is introduced in detail. It should be noted that the attached Fig.10 The image generating device in the present application is used to execute Figure 2-Figure 7 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2-Figure 7 The embodiment shown.

[0109] See also Fig.10 , which shows a schematic diagram of the structure of an image generation device provided by an exemplary embodiment of the present application. The image generation device can be implemented as all or part of the device through software, hardware or a combination of both. The device 1 includes a data image acquisition module 11, a data acquisition module 12, a priority confirmation module 13, and a cover confirmation module 14.

[0110] The image acquisition module 11 is used for acquiring video frame images as a first cover image set within the duration of the event when an event trigger is detected;

[0111] A data acquisition module 12, used for acquiring object data and screen change data corresponding to each first cover image in the first cover image set;

[0112] A priority confirmation module 13, used to confirm the priority of the first cover image based on the object data and the screen change data;

[0113] The cover confirmation module 14 is used to confirm the second cover image of the event based on the priority of each of the first cover images, and send the second cover image to the client. The second cover image is used by the client to generate a second cover image set based on the second cover image, and classify the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene.

[0114] Optionally, the priority confirmation module 13 is specifically used to confirm whether a target object exists in the first cover image based on the object data;

[0115] If yes, determining the priority of the first cover image based on the target object data corresponding to the target object in the object data;

[0116] If not, the priority of the first cover image is confirmed based on the screen change data.

[0117] Optionally, the priority confirmation module 13 is specifically used to obtain object evaluation data of the first cover image based on the target object data;

[0118] The priority of the first cover image is confirmed based on the object evaluation data.

[0119] Optionally, the priority confirmation module 13 is specifically configured to obtain an object image quality score of the first cover image when there is set feature data in the target object data;

[0120] When the set feature data does not exist in the target object data, the object recognition confidence of the first cover image is obtained.

[0121] Optionally, the priority confirmation module 13 is specifically used to confirm the priority of the target object when there are multiple target objects in the first cover image;

[0122] Based on the object data, object evaluation data of the target object with the highest priority in the first cover image is confirmed.

[0123] Optionally, the priority confirmation module 13 is specifically used to confirm the change area ratio of the first cover image and the historical moment image based on the screen change data, and the historical moment image is an image at a moment before the first cover image is captured;

[0124] The priority of the first cover image is determined based on the changed area ratio.

[0125] See also Fig.11 , which shows a schematic diagram of the structure of an image generating device provided by an exemplary embodiment of the present application. Fig.11 The image generating device in the present application is used to execute Figure 8-Figure 9 The method of the embodiment shown. The image generation device can be implemented as all or part of the device through software, hardware or a combination of both. The device 2 includes a cover acquisition module 21 and a cover classification module 22.

[0126] The cover acquisition module 21 is used to acquire the second cover image of each event captured by the camera and generate a second cover image set, wherein the second cover image is obtained by the camera based on the object data and the screen change data corresponding to each first cover image in the first cover image set corresponding to the event;

[0127] The cover classification module 22 is used to classify each of the second cover images in the second cover image set based on the scene, and obtain a scene cover image set corresponding to each scene.

[0128] Optionally, the cover classification module 22 is further used to obtain, from the camera, a target event video associated with each scene cover image in the scene cover image set;

[0129] A video display control is generated based on the scene cover image and the target event video, and the video display control is used to receive a scene cover image selection instruction from a user to play the target event video corresponding to the scene cover image.

[0130] It should be noted that, when the image generation device provided in the above embodiment executes the image generation method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image generation device provided in the above embodiment and the image generation method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0131] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0132] The present application also provides a computer storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned Figure 2-Figure 9 The image generation method of the embodiment shown in the figure can be specifically executed by referring to Figure 2-Figure 9 The specific description of the illustrated embodiment will not be repeated here.

[0133] Please refer to Fig.12 , which shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0134] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the terminal 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user pages, and applications; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0135] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium (Non-Transitory Computer-Readable Storage Medium). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems.

[0136] The memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, the requirements for system resources in different application scenarios in the same third-party application are also different. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0137] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0138] The input device 130 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output commands or data, and includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays.

[0139] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0140] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0141] exist Fig.12 In the electronic device shown, the processor 110 may be used to call a computer application stored in the memory 120 and specifically perform the following operations:

[0142] When an event trigger is detected, collecting video frame images within the duration of the event as a first cover image set;

[0143] Acquire object data and screen change data corresponding to each first cover image in the first cover image set;

[0144] determining a priority of the first cover image based on the object data and the screen change data;

[0145] A second cover image of the event is confirmed based on the priority of each of the first cover images, and the second cover image is sent to the client. The second cover image is used by the client to generate a second cover image set based on the second cover image, and the second cover image set is classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0146] In one embodiment, when the processor 110 determines the priority of the first cover image based on the object data and the screen change data, the processor 110 specifically performs the following operations:

[0147] Based on the object data, confirming whether a target object exists in the first cover image;

[0148] If yes, determining the priority of the first cover image based on the target object data corresponding to the target object in the object data;

[0149] If not, the priority of the first cover image is confirmed based on the screen change data.

[0150] In one embodiment, when the processor 110 determines the priority of the first cover image based on the target object data corresponding to the target object in the object data, the processor 110 specifically performs the following operations:

[0151] acquiring object evaluation data of the first cover image based on the target object data;

[0152] The priority of the first cover image is confirmed based on the object evaluation data.

[0153] In one embodiment, when the processor 110 acquires the object evaluation data of the first cover image based on the target object data, the processor 110 specifically performs the following operations:

[0154] When the target object data contains set characteristic data, obtaining an object image quality score of the first cover image;

[0155] When the set feature data does not exist in the target object data, the object recognition confidence of the first cover image is obtained.

[0156] In one embodiment, when the processor 110 confirms the object evaluation data of the first cover image based on the target object data, the processor 110 specifically performs the following operations:

[0157] When there are multiple target objects in the first cover image, confirming the priority of the target objects;

[0158] Based on the object data, object evaluation data of the target object with the highest priority in the first cover image is confirmed.

[0159] In one embodiment, when the processor 110 determines the priority of the first cover image based on the screen change data, the processor 110 specifically performs the following operations:

[0160] Determining the change area ratios of the first cover image and the historical moment image based on the screen change data, the historical moment image being an image at a moment before the first cover image is captured;

[0161] The priority of the first cover image is determined based on the changed area ratio.

[0162] In one embodiment, the processor 110 may further perform the following operations:

[0163] Acquire a second cover image of each event captured by the camera, and generate a second cover image set, wherein the second cover image is obtained by the camera based on object data and screen change data corresponding to each first cover image in the first cover image set corresponding to the event;

[0164] The second cover images in the second cover image set are classified based on the scene to obtain a scene cover image set corresponding to each scene.

[0165] In one embodiment, after performing scene-based classification processing on each of the second cover images in the second cover image set to obtain a scene cover image set corresponding to each scene, the processor 110 further performs the following operations:

[0166] Acquire, from the camera, a target event video associated with each scene cover image in the scene cover image set;

[0167] A video display control is generated based on the scene cover image and the target event video, and the video display control is used to receive a scene cover image selection instruction from a user to play the target event video corresponding to the scene cover image.

[0168] In an embodiment of the present application, when a camera detects an event trigger, it collects video frame images as a first cover image set within the duration of the event, obtains object data and screen change data corresponding to each first cover image in the first cover image set, confirms the priority of the first cover image based on the object data and screen change data, confirms the second cover image of the event based on the priority of each first cover image, and sends the second cover image to the client. The client generates a second cover image set based on the second cover image corresponding to each event, and classifies the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene. During the detection process of the camera, the collected video frame images are roughly screened to obtain the cover image corresponding to the triggering event, so that the client only needs to classify the cover image to obtain the cover image in different scenes, which reduces the client's computing burden, and can help users quickly locate the video information of interest by classifying the cover, thereby improving the efficiency of video playback. Furthermore, in the embodiment of the present application, by confirming whether there is a target object in the first cover image, when there is a target object in the first cover image, the target object data is obtained in the object data, and the priority of the first cover image is confirmed according to the target object data; when there is no target object in the first cover image, the priority of the first cover image is confirmed according to the screen change data, so that a cover image that is consistent with the video content and accurately summarizes the event can be efficiently generated to help users review quickly. Further, when confirming the priority of the first cover image based on the target object data, the object evaluation data of the first cover image is obtained according to the target object data, and the priority of the first cover image is confirmed based on the object evaluation data. By confirming the target object, it is only necessary to generate the priority according to the target object data, which can reduce the calculation amount of confirming the priority and improve the calculation speed; in addition, when there are multiple target objects, the target object data with the highest priority can be confirmed in the object data according to the priority of the target object, and the object evaluation data can be confirmed according to the target object data with the highest priority, so that the obtained cover image can include more target object information that users are concerned about. Furthermore, by confirming the proportion of the changed areas of the first cover image and the historical moment image based on the screen change data, and confirming the priority of the first cover image based on the proportion of the changed areas, images with larger screen changes in the first cover image can be identified, thereby improving the accuracy of outputting the second cover image.A second cover image set is generated by acquiring the second cover image of each event captured by the camera, and each second cover image in the second cover image set is classified based on the scene to obtain a scene cover image set corresponding to each scene, and a target event video associated with each scene cover image in the scene cover image set is acquired from the camera, and a video display control is generated based on the scene cover image and the target event video. Videos can be quickly found based on the classified cover images, which meets the user's need to quickly view videos and allows convenient viewing of the original video for complete viewing.

[0169] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0170] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image generation method, characterized in that: Applied to a camera, the method comprises: When an event trigger is detected, collecting video frame images within the duration of the event as a first cover image set; Obtaining object data and screen change data corresponding to each first cover image in the first cover image set, wherein the object data is the movable object appearing in the video frame image and the data obtained by analyzing the movable object; Based on the object data, confirming whether a target object exists in the first cover image; If so, obtaining object evaluation data of the first cover image based on the target object data, and confirming the priority of the first cover image based on the object evaluation data, wherein the object evaluation data is an object image quality score or an object recognition confidence; If not, determining the priority of the first cover image based on the screen change data; Determining a second cover image for the event based on the priority of each of the first cover images, sending the second cover image to the client, using the second cover image for the client to generate a second cover image set based on the second cover image, and classifying the second cover image set based on scenes to obtain a scene cover image set corresponding to each scene; Wherein, acquiring the object evaluation data of the first cover image based on the target object data includes: When there is set feature data in the target object data, obtaining an object image quality score of the first cover image, wherein the set feature data is a feature that needs to be paid attention to in the cover image, and the object image quality score is determined according to a quality index defined by a human visual system, and the quality index includes clarity, angle, and occlusion of the set feature data; When the target object data does not contain the set feature data, obtaining the object recognition confidence of the first cover image; The determining the priority of the first cover image based on the screen change data includes: Determining the change area ratios of the first cover image and the historical moment image based on the screen change data, the historical moment image being an image at a moment before the first cover image is captured; The priority of the first cover image is determined based on the changed area ratio.

2. The method according to claim 1, characterized in that The confirming the object evaluation data of the first cover image based on the target object data includes: When there are multiple target objects in the first cover image, confirming the priority of the target objects; Based on the object data, object evaluation data of the target object with the highest priority in the first cover image is confirmed.

3. An image generation method, characterized in that: Applied to the client, including: Acquire a second cover image of each event captured by a camera, and generate a second cover image set, wherein the second cover image is obtained by confirming the priority of each first cover image in the first cover image set corresponding to the event by the camera; the priority of the first cover image is confirmed based on the object data and screen change data corresponding to the first cover image; the object data is data obtained by analyzing a movable object appearing in a video frame image and the movable object; the object data is used to obtain object evaluation data of the first cover image based on the target object data if it is confirmed that a target object exists in the first cover image, and to confirm the priority of the first cover image based on the object evaluation data; the screen change data is used to confirm the priority of the first cover image based on the screen change data if it is confirmed that no target object exists in the first cover image; the object evaluation data is an object image quality score or an object recognition confidence; Classify each of the second cover images in the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene; Among them, the object image quality score of the first cover image is obtained when there is set feature data in the target object data, the set feature data is the feature that needs to be paid attention to in the cover image, and the object image quality score is determined according to the quality indicators defined by the human visual system, and the quality indicators include the clarity, angle, and occlusion of the set feature data; the object recognition confidence of the first cover image is obtained when there is no set feature data in the target object data; the screen change data is used to indicate the proportion of the changed area between the first cover image and the historical moment image, and the historical moment image is the image of the previous moment before the first cover image is collected.

4. The method according to claim 3, characterized in that After classifying each of the second cover images in the second cover image set based on the scene to obtain a scene cover image set corresponding to each scene, the method further includes: Acquire, from the camera, a target event video associated with each scene cover image in the scene cover image set; A video display control is generated based on the scene cover image and the target event video, and the video display control is used to receive a scene cover image selection instruction from a user to play the target event video corresponding to the scene cover image.

5. An image generating device, characterized in that: The device comprises: An image acquisition module, configured to acquire video frame images as a first cover image set within a duration of the event when an event trigger is detected; A data acquisition module, used to acquire object data and screen change data corresponding to each first cover image in the first cover image set, wherein the object data is the movable object appearing in the video frame image and the data obtained by analyzing the movable object; A priority confirmation module, used for confirming the priority of the first cover image based on the object data and the screen change data; a cover confirmation module, configured to confirm a second cover image of the event based on the priority of each of the first cover images, and send the second cover image to a client, wherein the second cover image is used by the client to generate a second cover image set based on the second cover image, and classify the second cover image set based on scenes to obtain a scene cover image set corresponding to each scene; The priority confirmation module is specifically used to confirm whether there is a target object in the first cover image based on the object data; if so, obtain object evaluation data of the first cover image based on the target object data, and confirm the priority of the first cover image based on the object evaluation data, wherein the object evaluation data is an object image quality score or an object recognition confidence; if not, confirm the priority of the first cover image based on the screen change data; When the priority confirmation module is used to obtain the object evaluation data of the first cover image based on the target object data, it is specifically used to obtain the object image quality score of the first cover image when there is set feature data in the target object data, the set feature data is the feature that needs to be paid attention to in the cover image, and the object image quality score is determined according to the quality index defined by the human visual system, and the quality index includes the clarity, angle, and occlusion of the set feature data; when there is no set feature data in the target object data, the object recognition confidence of the first cover image is obtained; When the priority confirmation module is used to confirm the priority of the first cover image based on the screen change data, it is specifically used to confirm the change area ratio of the first cover image and the historical moment image based on the screen change data, and the historical moment image is the image at the previous moment before the first cover image is captured; the priority of the first cover image is confirmed based on the change area ratio.

6. An image generating device, characterized in that: The device comprises: A cover acquisition module, used for acquiring a second cover image of each event captured by a camera, and generating a second cover image set, wherein the second cover image is obtained by the camera based on the priority confirmation of each first cover image in the first cover image set corresponding to the event; the priority of the first cover image is confirmed based on the object data and screen change data corresponding to the first cover image; the object data is the movable object appearing in the video frame image and the data obtained by analyzing the movable object; the object data is used for obtaining object evaluation data of the first cover image based on the target object data if it is confirmed that there is a target object in the first cover image, and confirming the priority of the first cover image based on the object evaluation data; the screen change data is used for confirming the priority of the first cover image based on the screen change data if it is confirmed that there is no target object in the first cover image; the object evaluation data is the object image quality score or the object recognition confidence; A cover classification module, used for classifying each of the second cover images in the second cover image set based on the scene, to obtain a scene cover image set corresponding to each scene; Among them, the object image quality score of the first cover image is obtained when there is set feature data in the target object data, the set feature data is the feature that needs to be paid attention to in the cover image, and the object image quality score is determined according to the quality indicators defined by the human visual system, and the quality indicators include the clarity, angle, and occlusion of the set feature data; the object recognition confidence of the first cover image is obtained when there is no set feature data in the target object data; the screen change data is used to indicate the proportion of the changed area between the first cover image and the historical moment image, and the historical moment image is the image of the previous moment before the first cover image is collected.

7. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer program, wherein the computer program is suitable for being loaded by the processor and executing the steps of the method according to any one of claims 1 to 4.

8. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images