Cover generation method and device, equipment and medium

Abstract

The embodiment of the invention provides a cover generation method and device, equipment and a medium. According to the embodiment of the invention, the method comprises the steps: obtaining alternative video frames in a to-be-processed house video; the alternative video frame is used for presenting at least one frame of image of the house information; selecting a person image from the alternative video frame, and extracting human body information from the person image; wherein the human body information comprises contour information and face information; and selecting one frame from the alternative video frames as a cover by using the contour information and / or the face information. And when the video frame is selected, the user does not need to participate, so that the workload of the user can be effectively reduced. When the target video frame is selected, the contour information and the face information are comprehensively utilized for selection, and the target video frame meeting the cover display requirement can be effectively selected. Namely, the cover can display the house information while meeting the image requirement of the display personnel, so that the house buying user can intuitively know the broker and house information through the cover.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a cover generation method, device, equipment and medium. Background Art

[0002] With the development of computer technology, more and more fields have played an important role. For example, real estate promotion is carried out online. In order to facilitate users to better understand the housing situation, a complete house video will be recorded for users.

[0003] On the page displaying the house information, a series of house information will be displayed in a list or other manner. In the introduction of each house information, a more representative cover will be set to better attract users to click on the house information, thereby increasing the click-through rate of the house information. In the prior art, brokers are usually required to maintain the house information. Generally speaking, brokers are required to select a frame from the house video as the cover. In addition, some will edit the cover to make it present a better effect, such as adding introductory text, adding broker photos, etc. The seemingly simple presentation effect requires brokers to pay a lot of time costs. Therefore, a solution that can quickly generate video covers is needed. Summary of the invention

[0004] Multiple aspects of the present application provide a cover generation method, device, equipment and medium to solve the technical problems of complex and low efficiency cover generation process.

[0005] In a first aspect, an exemplary embodiment of the present application provides a cover generation method, which is applied to a server side, and the method includes:

[0006] Acquire at least one candidate video frame containing a house image and a person image in the house video to be processed;

[0007] Extracting human body contour information and facial status information of a person in the person image of the at least one candidate video frame;

[0008] Using the human body contour information to determine whether a person in the at least one candidate video frame appears within a legal area; and

[0009] Using the facial status information, determining whether the person in the at least one candidate video frame meets the status condition;

[0010] In response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition, extracting the candidate video frame from the house video to be processed as the target video frame;

[0011] The extracted target video frame is added to the cover position of the house video to be processed.

[0012] In a second aspect, an exemplary embodiment of the present application provides a cover generation device, the device comprising:

[0013] An acquisition module, used for acquiring at least one candidate video frame containing a house image and a person image in the house video to be processed;

[0014] An extraction module, used to extract the human body contour information and facial state information of the person in the person image of the at least one candidate video frame;

[0015] A judgment module, used to judge whether a person in the at least one candidate video frame appears in a legal area by using the human body contour information; and

[0016] The judging module is further configured to use the facial status information to judge whether the person in the at least one candidate video frame meets the status condition;

[0017] The extraction module is further configured to extract the candidate video frame as the target video frame from the house video to be processed in response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition;

[0018] The adding module is used to add the extracted target video frame to the cover position of the house video to be processed.

[0019] In a third aspect, an exemplary embodiment of the present application further provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program to execute the steps in the above-mentioned cover generation method.

[0020] In a fourth aspect, an exemplary embodiment of the present application provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned cover generation method.

[0021] In this embodiment, after obtaining the house video shot in advance, an alternative video frame is selected according to the established rules (for example, according to the established periodic interval of 1 second). Further, the content contained in the selected alternative video frame is identified, including the house image. In addition, it is also necessary to distinguish whether the alternative video frame contains a person image. If it is contained, the video frame containing the person image is preferentially selected as the alternative video frame. Then, the human body information of the person image is extracted from the alternative video frame. The human body information includes: human body contour information and facial state information. According to the human body contour information and / or facial state information, one of the frames is selected from the selected live broadcast according to the established rules to generate the cover. Through the above scheme, when selecting the video frame, the user (for example, the agent) does not need to participate, which can effectively reduce the user's workload. When selecting the target video frame, the target video frame that meets the cover display requirements can be effectively selected by comprehensively utilizing the human body contour information and facial state information. That is, the cover can meet the requirements of displaying the person image while also being able to display the house image, so that the house buyer can intuitively understand the agent and house information through the cover. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A schematic diagram of a flow chart of a cover generation method provided in an embodiment of the present application;

[0024] Figure 2 A schematic flow chart of a method for selecting a cover for illustrating an embodiment of the present application;

[0025] Figure 3a , 3b A schematic diagram of a prohibited display area for illustrating an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of a cover generation device provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a cover generation scheme illustrated in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0030] With the development of network technology, transaction needs have shifted from offline to online. In real estate trading platforms, in order to enable home buyers to have a more intuitive and comprehensive understanding of house layout, apartment type and other related information, house videos are usually recorded. In some cases, the agent will introduce the house information in the video. In other cases, if there is no agent to introduce in the video, the video content can be introduced by post-dubbing or subtitles. In the prior art, the house information is usually maintained by the agent, and the agent is also required to select one frame from the house video as the cover of the house information or the cover of the house video. In some cases, the agent has to add the cover according to the template designed in advance by the designer. This process takes a lot of time and energy for the agent. Therefore, a solution is needed to generate the cover of the house video simply and quickly.

[0031] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0032] Figure 1 The following is a flow chart of the cover generation method provided in the embodiment of the present application. Figure 1 As shown, the cover generation method provided in the embodiment of the present application can be run on a server side, which can be a local server or a cloud server. The method specifically includes the following steps:

[0033] Step 101: Obtain at least one candidate video frame containing a house image and a person image in a house video to be processed.

[0034] Step 102: extracting body contour information and facial status information of a person in the person image of the at least one candidate video frame.

[0035] Step 103: Using the human body contour information, determine whether a person in the at least one candidate video frame appears within a legal area. And,

[0036] Step 104: Use the facial status information to determine whether the person in the at least one candidate video frame meets the status condition.

[0037] Step 105: In response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition, the candidate video frame is extracted from the house video to be processed as the target video frame.

[0038] Step 106: Add the extracted target video frame to the cover position of the house video to be processed.

[0039] In real estate (including office buildings and residences) online transaction platforms (e.g., house rental, house sale, etc.), house videos will be added to the list of house information provided by the real estate transaction platform. Of course, other forms may also be used, such as displaying house images in a nine-square grid.

[0040] Regardless of the method used to display the house information, the video cover will be displayed if the user does not click to play the house video. In other words, the video cover is the first thing that users see when representing the house information, and the quality of the cover directly affects the user's click rate and the effect of attracting users. Therefore, after the server obtains the house video (for example, the user uploads the house video to the server after recording it), it will identify the content of the video frame frame by frame, and these video frames involved in the identification are called alternative video frames. The alternative video frames include house images, and some video frames also include images of real estate agents. When using alternative video frames to select video frames suitable for the cover, it may be necessary to identify multiple alternative video frames and then determine the required target video frame. When extracting the target video frame, the alternative video frame can be obtained by copying it, or by taking a screenshot or screenshot of the alternative video frame.

[0041] Here, each candidate video frame corresponds to an image frame. When selecting candidate video frames, you can select according to the period, for example, select one frame of image as the candidate video frame every 1 second. If the original recorded house video is too long, you can also choose to select one frame of image as the candidate video frame every 2 seconds. Therefore, when selecting candidate video frames, you can determine the selection period or selection frequency according to the length of the house video. The longer the house video is, the longer the selection period (the lower the selection frequency); conversely, the shorter the house video is, the shorter the selection period (the higher the selection frequency).

[0042] After selecting the candidate video frames, the content contained in the candidate video frames will be analyzed. During the video recording process, when the camera follows the broker (or real estate agent) to move in the house, it must capture the interior of the house clearly and follow the broker's movement, which leads to the diversity of the content presented in the video frames. The diversity mentioned here can be understood as some candidate video frames contain images of people, some candidate video frames do not contain images of people, and the house images in the candidate video frames also change accordingly with the position and angle of the camera.

[0043] In some cases, the agent wants to show his or her image in the cover, so the video frame containing the person image will be preferred as the alternative video frame. On the contrary, if some agents are more introverted, or want to present more house images (such as the spatial layout of the house, house decoration, etc.), the agent's person image may not appear in the video, or a video frame without a person image may be selected as the alternative video frame and cover. Alternatively, an image with a small proportion of the person image in the video frame may be selected as the alternative video frame and cover.

[0044] After selecting an alternative video frame containing a person's image, human body information will be extracted from the person's image in the alternative video frame. The human body information mentioned here includes: human body contour information and facial status information of the person. The human body contour information mentioned here can be understood as human body contour information composed of human body shape or human body skeleton key points. Based on the human body contour information, the minimum area of ​​the person in the alternative video frame can be outlined. In addition, based on the contour information, the contour coordinate position of each contour key point in the alternative video frame can also be determined. Using the human body contour information, the status of the person in the alternative video frame can be specifically analyzed, which will be explained in the following embodiments and will not be repeated here.

[0045] In practical applications, even if a person image is included, further analysis is required to determine whether the image is of a designated person, for example, whether the image includes the agent's image or other persons such as the homeowner. If the image includes images of persons other than the agent, the person in the candidate video frame needs to be further identified. If the image includes irrelevant persons such as the homeowner, the video frame is discarded. In other words, the video frame cannot be used as a candidate video frame.

[0046] After obtaining the candidate video frames and the human information (including human outline information and facial status information) contained therein in the above manner, the human outline information and / or facial status information can be further used to select a frame from the candidate video frames as the cover. The selection is based on actual needs, including the requirements for the presentation effect of personnel images and the requirements for the presentation effect of house images. The judgment method for these actual needs can use the human outline information and facial status information to comprehensively analyze the candidate video frames in order to determine the target video frame for generating the cover. The process of generating the cover does not require the participation of users (for example, agents), which can effectively reduce the workload of personnel in making video covers and improve work efficiency.

[0047] In one or more embodiments of the present application, Figure 2 This is a flow chart of a method for determining a person's appearance area, which is illustrated in an embodiment of the present application. Figure 2 As shown, the method of using the human body contour information to determine whether a person in the at least one candidate video frame appears in a legal area specifically includes the following steps:

[0048] Step 201: Identify the house image contained in the candidate video and mark the prohibited display area in the candidate video frame.

[0049] Step 202: Determine the coordinate range corresponding to the prohibited display area.

[0050] Step 203: judging whether key points of the human body contour appear within the coordinate range of the prohibited display area according to the human body contour information.

[0051] Step 204: If the judgment result is that the human body contour key point does not appear in the coordinate range of the prohibited display area, use multiple human body contour key points in the human body contour information to determine whether the display range of the person image in the alternative video frame at least includes the upper body range of the person.

[0052] The specific judgment process of judging whether the key points of the human body contour appear in the coordinate range of the prohibited display area as described in step 203 is as follows:

[0053] Step 2031: Determine the human body contour coordinates corresponding to the multiple contour key points in the human body contour information.

[0054] Step 2032: If the number of the contour key points falling into the prohibited display area determined based on the human body contour coordinates is greater than a quantity threshold, and / or if the maximum distance of the edge of the contour key points falling into the prohibited display area determined based on the contour coordinates is greater than a distance threshold, then the person image is displayed in the prohibited display area.

[0055] Step 2033: If it is determined based on the contour coordinates that the number of contour key points falling into the prohibited display area is not greater than the number threshold, and the maximum distance between the contour key points falling into the prohibited display area and the edge is not greater than the distance threshold, then the person image is not displayed in the prohibited display area.

[0056] In actual applications, if the user wants to get a cover that can fully reflect the image of the agent, the contour information and facial information will be used to comprehensively select a suitable target image. The details are as follows:

[0057] From the contour information returned by OpenPose or other human detection algorithms, extract the contour coordinates of all contour key points that constitute the person's contour. These contour key points may include various joints of the human body, such as shoulders, elbows, wrists, hips, knees, ankles, etc., and may include additional points to more accurately depict the human body contour. In addition, when obtaining the key points of the upper body, they can be marked more closely. For example, the key points of the top of the head, the key points of the shoulders, the key points of the upper limbs, and the key points of the upper torso, so as to better mark the contour of the person in the video frame. There are also key points of the lower limbs and torso below.

[0058] Based on the extracted key point coordinates, the smallest polygon or convex hull that can represent the person's outline is calculated using geometric methods (such as polygon approximation, convex hull algorithm, etc.). Then, the vertex coordinates of this polygon or convex hull are determined, and these vertex coordinates will be used for subsequent comparison with the prohibited display area. The calculated human body outline coordinates are subjected to necessary optimization processing, such as removing noise points, smoothing the outline, etc., to improve the accuracy of subsequent judgments.

[0059] Define the boundaries of the prohibited display area based on the resolution and layout requirements of the video frame. These boundaries are usually based on the edges of the video frame, but can also include specific areas within the frame (such as areas reserved for logos, text, or other important elements). If the video frame and the definition of the prohibited display area use different coordinate systems (such as pixel coordinates, percentage coordinates, etc.), coordinate conversion is required to ensure that they are compared in the same coordinate system. Record the defined prohibited display area in the form of coordinates, usually a set of coordinates of one or more rectangular or complex polygonal areas.

[0060] Compare the calculated human body contour coordinates (especially the vertex coordinates of the contour boundary) with the defined prohibited display area coordinates. Use geometric algorithms (such as ray method, bounding box intersection method, etc.) to detect whether the human body contour intersects with the prohibited display area. If there is at least one contour coordinate point located in the prohibited display area, or the boundary of the human body contour intersects with the boundary of the prohibited display area, it is considered that the person image appears in the prohibited display area. If it is detected that the person image appears in the prohibited display area, the result is recorded and may trigger subsequent processing procedures (such as skipping the current frame, marking as not meeting the conditions, etc.). If it does not appear in the prohibited display area, continue with the subsequent judgment steps.

[0061] For example, Figure 3a , 3b This is a schematic diagram of a prohibited display area for illustrating an embodiment of the present application. This schematic diagram is an interface schematic diagram, which can be a client interface of a mobile phone, a client interface of a computer, or an interface of other electronic devices. Figure 3a As can be seen in the figure, if the person image is in the center of the video frame, the person image can be well presented. If the person image is at the edge, or the person is standing sideways, the person image cannot be well presented. Or, Figure 3b As shown in Figure 2, people occupy more than half of the video frame area, leaving no more space to display house information. Figure 3a and 3b In the illustrated embodiment, the upper third of the video frame image is marked as a prohibited display area, including the upper left corner and the upper right corner. It should be noted that the position of the prohibited display area can be changed according to different needs. For example, when you want to highlight the furniture in the house, the furniture in the alternative video frame is identified, and the area where the furniture appears is marked as a prohibited display area. Since the camera and the personnel (for example, the agent) are always moving during the video shooting process, the prohibited display area is also always changing dynamically. After determining the prohibited display area, the contour coordinate information is further compared with the coordinate information of the prohibited display area to finally determine whether the person appears in the prohibited display area. If the image of the person appears in the prohibited display area, the alternative video frame is unqualified and cannot be used as a cover. If the image of the person does not appear in the prohibited display area, the alternative video frame meets a condition as a target video frame.

[0062] As described in step 104, using the facial state information to determine whether the facial state information of the person in the at least one candidate video frame meets the state condition specifically includes: step 2031: determining the facial key points constituting the facial contour in the facial state information and the corresponding facial coordinate range; determining whether the area ratio of the facial coordinate range in the candidate video meets the first area condition. Step 2032: determining the eye key points constituting the eye contour in the facial information and the corresponding eye coordinate range; determining whether the area ratio of the eye coordinate range in the facial coordinate range meets the second preset condition.

[0063] When calculating facial information, the openpose algorithm can also be used together with the contour information to calculate it. Using the calculated facial information, determine the position and size of the face area in the candidate video frame. The face frame is usually a rectangle that contains the main part of the face. Then, calculate the proportion of the face area in the entire video frame. This can be achieved by calculating the ratio of the area of ​​the face frame to the area of ​​the video frame. Specifically, the area of ​​the face frame can be obtained by multiplying its width and height, and the area of ​​the video frame is the product of its width and height. Compare the calculated face area ratio with the preset first ratio threshold. If the ratio is less than the first ratio threshold, it means that the space occupied by the face is too small, and the facial feature information of the agent may not be fully and clearly displayed, and the display effect of the personnel image is not good. If the ratio is greater than or equal to the first ratio threshold (for example, 30%), it is considered that the personnel image can be displayed more clearly at this time, and then continue with the subsequent judgment steps.

[0064] In addition to evaluating the face ratio, it is also necessary to ensure that the agent can open his eyes. Based on the eye key point information returned by the facial key point detection algorithm (such as OpenPose, D lib, etc.), determine the position and size of the eye area in the face area. The eye area can be approximately represented by a rectangle or ellipse formed by connecting the key points of the eye (such as the corner of the eye, the center of the pupil, etc.). Next, calculate the proportion of the eye area in the face area. This can be achieved by calculating the ratio of the area of ​​the eye area to the area of ​​the entire face area. The area of ​​the eye area can be obtained by multiplying its width and height (or the area formula of an ellipse). Compare the calculated eye area ratio with the preset second ratio threshold. If the ratio is less than the second ratio threshold, it means that the eye area is not displayed clearly in the face or may be in a closed state, so it is judged as not meeting the conditions. This may mean that the photo quality is poor, the shooting time is inappropriate, or the person is in an inactive state. If the ratio is greater than or equal to the second ratio threshold, the display of the eye area is considered to be clear and meets the requirements, and the subsequent judgment steps can be continued or finally determined as a qualified cover candidate.

[0065] In an optional solution, when setting the first preset condition, not only the minimum proportion is set, but also the maximum proportion needs to be set. If the proportion is greater than the maximum proportion threshold, it means that the face occupies too much space, which may block or affect the display of other important information (such as housing information), so it is judged as not meeting the conditions. If the proportion is less than or equal to the maximum proportion (for example, 60%), and greater than or equal to the first proportion threshold (for example, 30%), then the face image can be clearly displayed without occupying too much display space or blocking too much housing information, and the subsequent judgment steps will continue.

[0066] The human body contour information contains a plurality of contour key points, wherein the plurality of contour key points include at least one of a top of the head key point, a shoulder key point, an upper limb key point, an upper torso key point, and a lower torso key point.

[0067] As described in step 204, using a plurality of human body contour key points in the human body contour information to determine whether the display range of the person image in the candidate video frame at least includes the upper body range of the person, specifically includes the following steps:

[0068] Step 2041: Determine key points of the outline representing the person based on the outline information.

[0069] Step 2042: Determine whether the contour key points include the top of the head key points, the shoulder key points, the upper limb key points, and the upper torso key points representing the upper body, and do not include the lower torso key points representing the lower body of the person.

[0070] Step 2043: If yes, the display range of the person image in the alternative video frame is the upper body range of the person.

[0071] First, the contour information in the candidate video frame is preprocessed, such as denoising and smoothing, to improve the accuracy (that is, confidence) of subsequent key point detection. The contour key point detection algorithm (such as circle detection and ellipse detection based on Hough transform, or deep learning models such as Pose Est imation model) is used to identify the contour key points of the person in the image. These key points usually include the main joints of the human body, such as the head, shoulders, elbows, wrists, hips, knees, ankles, etc.

[0072] Among all the detected contour key points, select those that clearly represent the contour of the person, especially those that can clearly define the upper body of the person, such as shoulders, neck, etc. Determine whether the contour key points contain the target key points representing the upper body

[0073] Identify which contour key points are considered as target key points representing the upper body. Usually, these key points include the shoulder points (or the midpoint of the shoulder area), the neck point, etc., which can clearly define the range of the upper body. Traverse the filtered contour key point set to check whether it contains at least one or more predefined target key points. If at least one target key point is found, it is considered that the human body contour information contains the key information representing the upper body; otherwise, it is considered that the contour information is insufficient to represent the complete upper body.

[0074] If the target key point representing the upper body is indeed included in the contour key point set, this step is performed. Confirm that the contour information of the person's image meets the third preset condition, that is, the upper body of the person is indeed displayed in the image. This can be used as a prerequisite for further processing (such as cover selection, face recognition, etc.). Once it is confirmed that the display range in the alternative video frame is the upper body range of the person, subsequent processing steps such as image cropping, scaling, quality assessment, etc. can be performed as needed to optimize the image display effect or meet specific application requirements.

[0075] It should be noted that when judging the range of the upper body of a person, a set of key points of the human body outline can be used for judgment. That is, when it is identified that the image of the person in the alternative video frame contains all the key points of the outline, it is considered that the display range in the alternative video frame is the range of the upper body of the person. Of course, the size of the upper body image can also be limited. For example, the key points of the outline that must be included can be set, and the key points of the outline that are prohibited from appearing (for example, the key points of the outline representing the lower body) can also be set. This can meet the display requirements of the upper body while preventing the image of the person from occupying too much content space of the video frame.

[0076] After the above-mentioned steps are performed, the extracted target video frame is added to the cover position of the house video to be processed, including:

[0077] In response to a judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition, acquiring depth information corresponding to the person image in the candidate video frame;

[0078] Using the depth information, determining whether the distance relationship between the person and the camera shooting the video of the house meets the distance condition;

[0079] If it meets the requirements, the candidate video frame is determined as the target video frame.

[0080] In order to more clearly describe the use of the depth estimation model to estimate the depth of the video frame, and to evaluate the width of the video frame based on the key point information of the human skeleton (especially the spatial depth features after excluding the influence of the human body), the specific process is as follows:

[0081] First, obtain the RGB image of the video frame in the house video. Use a pre-trained depth estimation model (such as a model based on a convolutional neural network CNN) to process the input video frame and generate the corresponding depth map. In the depth map, the gray value of each pixel represents the depth of the position in the scene, that is, the larger the gray value, the closer the object corresponding to the pixel is to the camera; conversely, the smaller the gray value, the farther the distance. Output the depth map of the video frame.

[0082] Use human pose estimation models (such as OpenPose, HRNet, etc.) to detect key points of the human skeleton (such as head, shoulders, hips, etc.) in the depth map. Output the coordinate information of the key points of the human skeleton.

[0083] Calculate the smallest rectangle that can enclose the human body based on the key points of the human skeleton (such as shoulder width and hip height). Select the midpoint of the shoulder or hip as the midpoint of the lower boundary of the rectangle, extend 200 pixels upward according to the average human body proportion as the upper boundary, and extend the left and right boundaries to the left and right edges of the frame respectively. Output the coordinates of the human body enclosing rectangle.

[0084] After excluding the human body enclosing rectangle, calculate the remaining area. Count the percentage of pixels in the remaining area whose depth value is greater than a given threshold, which indicates that there are many close objects or scene details in the area. Calculate the grayscale mean of the entire depth map to reflect the depth distribution of the entire scene.

[0085] Calculate the depth threshold ratio (weight 50%), and use the calculated ratio directly as part of the score. Considering that a lower mean usually means more distant scenes, the depth mean (weight 50%) can be normalized or reversed (such as 1-normalized mean) to make it correspond to a better picture width with a low score. Add the above two scores with a weight of 50%, and take the weighted sum to get the final picture width score. The lower the score, the wider the depth distribution of the remaining area after excluding the human body, and the better the sense of space.

[0086] Output the width score of the video frame, which can be used in application scenarios such as video analysis, scene understanding, or video quality assessment. Through the above steps, the depth estimation and human posture estimation technology can be systematically used to perform depth analysis and quantitative assessment of the width of the video frame.

[0087] In one or more embodiments of the present application, generating the cover using the target video frame includes:

[0088] Determining whether the target video frame contains text content;

[0089] If there is text content, the real estate information content is drawn on the cover after erasing the text content according to the text generation rules; wherein the text generation rules include: at least one of text size, text content and text effect; or,

[0090] If there is text content, the real estate information content is overlaid on the text content according to the display specifications of the text content; wherein the display specifications include: at least one of display area size, display area coordinates and display transparency.

[0091] It should be noted that the text effects mentioned here include the text’s shade, layout, font type, display effects, etc.

[0092] In actual applications, some videos are equipped with subtitles. These subtitles are used to explain the video content. However, when generating the cover, these subtitles should not appear in the text. Therefore, after determining the target video frame, it will be determined whether the frame contains text content. When identifying the text content, a strategy should be adopted to avoid mistakenly identifying the content appearing in the video as the text content of the subtitle. Therefore, when judging, the specified area (generally the bottom edge of the video frame) is identified, and when identifying, it is determined whether each text is neatly arranged along the bottom edge. In addition, it is also necessary to determine whether these text contents will move with the lens (that is, when the house information or the image of the person in the video moves, the subtitles do not move). If it is the text content of the subtitle, it will not move with the lens, and if it is the text in the video, it will move with the content in the video. If there is no text, do not do any processing, and you can directly display the subtitles at the required position as needed.

[0093] If there is text content in the subtitles, the text content needs to be further processed. There are two ways to process it: one is to delete the original text content of the subtitles, and the other is to directly overwrite the original text content of the subtitles. The following will explain them separately.

[0094] Assuming that the solution of deleting the text content of the original subtitle and then drawing the real estate information content is adopted, first, it is necessary to identify whether there are subtitles in the video frame. This can be achieved through image recognition technology (such as OCR) or knowing the subtitle position in advance (such as a fixed position or a subtitle file). Set text generation rules according to requirements, including text size, text content (such as real estate address, price, area, etc.), and text effects (such as font, color, shadow, etc.). These rules can be flexibly adjusted according to different real estate information.

[0095] Before drawing new property information, the original subtitle content needs to be erased. This can be achieved by drawing a rectangular block of the same or similar color as the background in the subtitle area, or using image restoration technology to fill the subtitle area. Draw the property information content at the specified position of the cover or video frame according to the set text generation rules. Ensure that the information is clear and readable without affecting the overall appearance of the video or cover. Due to the limited amount of house text content added, the property information mentioned here can be simple and clear information, such as community name, full five years and only, house price, etc. Fine-tune the drawn property information, such as adjusting the position, size or effect, to ensure that it meets the design requirements. At the same time, a preview check can be performed to ensure that the processing results meet the requirements.

[0096] Assume that a scheme of drawing real estate information content by overwriting the text content of the original subtitle is adopted. First, the display specifications of the original subtitle content need to be obtained, including the display area size, display area coordinates, and display transparency, etc. This information can be obtained by analyzing the subtitle file or image recognition technology.

[0097] Prepare the real estate information content according to the requirements, including text content, fonts, colors, etc. Make sure that this information is compatible with the display specifications of the original subtitles. Since the amount of house text content added is limited, the real estate information here can be simple and clear information, such as community name, full five years of ownership, house price, etc.

[0098] On the display area of ​​the original subtitles, according to the acquired display specifications and the prepared real estate information content, overlay drawing is performed. The display effect of the new and old content can be balanced by adjusting the transparency to make the real estate information more prominent. Avoid the original text content that is covered from being displayed through the real estate information, which will bring a bad visual experience to users. If the real estate information overlaps or conflicts with other elements in the video or cover, appropriate adjustments need to be made. For example, move the position of the real estate information, adjust its size, or change its display method. After the overlay drawing is completed, perform a preview check to ensure that the real estate information is clearly visible and does not affect the overall effect of the video or cover. Make further adjustments and optimizations as needed.

[0099] It should be noted that before finding the target video frame that can be used to generate the cover, the execution process of the above steps is executed in a loop, and the loop execution of the above steps is stopped until the target video frame is found.

[0100] In actual applications, when shooting a video, you may shoot it horizontally and then play it vertically. When playing the video, there will be black areas above and below the video interface. When making a cover, this black area cannot appear. Therefore, you can choose to crop or rotate the video to obtain an alternative video frame. You can also discard the video frame.

[0101] In addition, when obtaining video frames from house videos, in order to more accurately determine whether the video frames have motion blur, the basic idea of ​​the optical flow algorithm can be used to determine:

[0102] First, load the video frame, make sure the video stream has been loaded, and have access to the current frame and the previous frame. Set a reasonable optical flow mean threshold based on the actual application scenario and test data. This threshold will be used to determine whether the inter-frame motion is too large, which may indicate motion blur.

[0103] In OpenCV, there are many optical flow algorithms to choose from, such as the Lucas-Kanade method (cv2.cal cOptica l F l owPyrLK), the Farneback method, etc. Considering that the Lucas-Kanade method performs well when processing sparse feature points, and the Farneback method is suitable for calculating dense optical flow fields, you can choose according to your needs. For motion blur detection, the Farneback method may be more suitable because it can provide more comprehensive inter-frame motion information.

[0104] Calculate the optical flow field between the current frame and the previous frame using the selected optical flow algorithm. This will generate a two-dimensional vector field of the same size as the previous frame, where each vector represents the displacement of the corresponding pixel between the two frames.

[0105] For each vector in the optical flow field, calculate the modulus of the optical flow vector (i.e. the size of the displacement). This can be achieved by calculating the L2 norm of the vector. Average the moduli of all optical flow vectors to get the mean of the optical flow field. This mean reflects the average movement distance of pixels between frames.

[0106] The calculated optical flow mean is compared with the preset threshold. If the mean is greater than the threshold, it is considered that the current frame has significant motion blur; otherwise, it is considered that the motion blur of the current frame is not obvious.

[0107] Dynamically adjusting the threshold according to changes in video content may help improve the accuracy of judgment. For example, when the video scene switches or the camera moves quickly, the threshold can be appropriately increased. In order to judge motion blur more robustly, you can consider using the average of the optical flow of multiple consecutive frames, rather than just based on the comparison of the current frame with the previous frame. This helps smooth noise and short-term fluctuations. In addition to optical flow information, other image features (such as edge blur, image contrast, etc.) can also be combined to comprehensively judge motion blur. Through the above steps, it is possible to more accurately judge whether the video frame has motion blur, so that the video frame with motion blur can be excluded to avoid adverse effects on subsequent video analysis. In other words, the video frame with motion blur can be excluded in advance to avoid useless analysis and waste of computing resources and time.

[0108] In practical applications, if no candidate video frame containing a person image is found, the candidate video frame is scored using a pre-trained scene scoring model; and a frame is selected from the candidate video frames as the cover according to the score value obtained by the scoring.

[0109] If no suitable target video frame is found through the scheme described above, or if no person image appears in the house video, an alternative video frame will be selected from video frames that do not contain person images.

[0110] Open the video file using a video processing library such as OpenCV. Read the video frame by frame until the end of the video. For each frame, perform the subsequent steps.

[0111] The video frame is filtered to determine whether there are large black areas above and below the screen. As an optional method, it is determined whether there are continuous areas in the horizontal or vertical direction of the video frame, and the pixel value of the continuous area does not exceed (0,0,0). If there are continuous areas, it is considered that there are black areas in the video frame.

[0112] Methods for determining whether a video frame is motion blurred (in addition to the methods mentioned above, you can also refer to existing algorithms, such as gradient-based methods): Calculate the gradient of each region in the frame. Count the degree and magnitude of the gradient direction change to determine whether it is a uniform change. If the gradient change is small and uniform, it may indicate that the image is blurred, and decide whether to exclude the frame based on the degree of blur.

[0113] As an alternative, you can use the YOLO model to determine whether there is a person in the video frame. Load the pre-trained YOLO model. Pre-process the video frame (such as resizing and normalizing). Use the YOLO model for inference and obtain the detection result. If no person is detected, keep the adapted frame.

[0114] Use the pre-trained Di no model as a feature extractor to extract deep features of the video frame. Multilayer Perceptron (MLP), the input is the output of the feature extractor, and the output is the probability value of the scene being a positive sample. This model can be trained independently or fine-tuned based on a large model. During the training process, frames are extracted from real videos and manually labeled as positive samples (such as scenes without people such as scenery and buildings) and negative samples (scenes with people or not meeting the requirements). Freeze the parameters of the Di no model. Only train the parameters of the classification head, using the cross entropy loss function. Use an appropriate optimizer (such as Adam) and learning rate strategy. Perform multiple rounds of training until the model performs stably on the validation set.

[0115] During use, the scene scoring model is used to infer the video frames filtered by the previous steps to obtain the scene score. The entire video is processed in a loop, and the above filtering and scoring steps are applied frame by frame. A record of the highest score and the corresponding video frame is maintained. After all frames are processed, the video frame with the highest score is selected as the cover of the video. The path of the best video frame is output or the image of the frame is directly output.

[0116] like Figure 4 A schematic diagram of the structure of a cover generation device provided in an embodiment of the present application. Figure 4 As can be seen in the figure, the device includes:

[0117] The acquisition module 41 is used to acquire at least one candidate video frame containing a house image and a person image in the house video to be processed.

[0118] The extraction module 42 is used to extract the human body contour information and facial state information of the person in the person image of the at least one candidate video frame.

[0119] The judging module 43 is used to judge whether a person in the at least one candidate video frame appears in a legal area by using the human body contour information. And,

[0120] The judgment module 43 is further configured to use the facial status information to judge whether the person in the at least one candidate video frame meets the status condition.

[0121] The extraction module 42 is further configured to extract the candidate video frame as the target video frame from the house video to be processed in response to a judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition.

[0122] The adding module 44 is used to add the extracted target video frame to the cover position of the house video to be processed.

[0123] Optionally, the judgment module 43 is used to identify the house image contained in the candidate video and mark a prohibited display area in the candidate video frame;

[0124] Determine the coordinate range corresponding to the prohibited display area;

[0125] According to the human body contour information, determining whether a key point of the human body contour appears within the coordinate range of the prohibited display area;

[0126] If the judgment result is that the human body contour key point does not appear within the coordinate range of the prohibited display area, multiple human body contour key points in the human body contour information are used to judge whether the display range of the person image in the alternative video frame at least includes the upper body range of the person.

[0127] Optionally, the judging module 43 is used to determine the human body contour coordinates corresponding to the multiple contour key points in the human body contour information;

[0128] If the number of the key points of the outline falling into the prohibited display area is greater than the number threshold according to the human body outline coordinates, and / or if the maximum distance of the edge of the key points of the outline falling into the prohibited display area is greater than the distance threshold according to the outline coordinates, then the person image is displayed in the prohibited display area;

[0129] If it is determined based on the contour coordinates that the number of contour key points falling into the prohibited display area is not greater than the number threshold, and the maximum distance between the contour key points falling into the prohibited display area and the edge is not greater than the distance threshold, then the person image is not displayed in the prohibited display area.

[0130] Optionally, the judgment module 43 is used to determine the facial key points constituting the facial contour in the facial state information and the corresponding facial coordinate range; and judge whether the area ratio of the facial coordinate range in the candidate video meets the first area condition;

[0131] Determine the eye key points constituting the eye contour in the facial information and the corresponding eye coordinate range; and determine whether the area ratio of the eye coordinate range in the facial coordinate range meets the second area condition.

[0132] Optionally, the multiple contour key points contained in the human body contour information include at least one of a top of the head key point, a shoulder key point, an upper limb key point, an upper torso key point, and a lower torso key point;

[0133] Optionally, the judgment module 43 is used to determine the key points of the outline representing the person based on the human body outline information; determine whether the key points of the outline include the key points of the top of the head, the key points of the shoulders, the key points of the upper limbs, and the key points of the upper torso, and do not include the key points of the lower torso representing the lower torso of the person;

[0134] If so, the display range of the person image in the alternative video frame is the upper body range of the person.

[0135] Optionally, the extraction module 42 is used to obtain depth information corresponding to the image of the person in the candidate video frame in response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition;

[0136] Using the depth information, determining whether the distance relationship between the person and the camera shooting the video of the house meets the distance condition;

[0137] If it meets the requirements, the candidate video frame is determined as the target video frame.

[0138] Optionally, the adding module 44 is used to determine whether the target video frame contains text content;

[0139] If there is text content, the real estate information content is drawn on the cover after erasing the text content according to the text generation rules; wherein the text generation rules include: at least one of text size, text content and text effect; or,

[0140] If there is text content, the real estate information content is overlaid on the text content according to the display specifications of the text content; wherein the display specifications include: at least one of display area size, display area coordinates and display transparency;

[0141] The target video frame containing the real estate information content is added to the cover position of the house video to be processed.

[0142] Optionally, the acquisition module 41 is used to score the candidate video frame using a pre-trained scene scoring model if no candidate video frame containing a person image is found;

[0143] According to the scoring value, a frame is selected from the candidate video frames as the cover.

[0144] For ease of understanding, the technical solution of this application will be described below through a complete embodiment. Figure 5 This is a schematic diagram of a cover generation scheme for illustrating an embodiment of the present application. Figure 5 As you can see, the specific generation process is as follows:

[0145] At the beginning, the user initiates a cover generation request, and then the server starts to extract video frames from the house video. When extracting, one frame can be extracted every 1 second. After the extraction is completed, the openpose algorithm is used to process the extracted candidate video frames to identify the human body information (contour information and facial information) in them. According to the recognition result, it is determined whether there is no person or multiple people. If yes, it is further determined whether there is no person. If not, the candidate video frame without people is saved, and then the candidate video frame without people is analyzed and enters the next loop. If there is and only one person, it is determined whether there is a black area in the video frame, whether it is a moving picture, and whether the position of the person image in the video frame appears in the prohibited display area. In addition, it is also determined whether the person image shows the upper body, and whether the face ratio and eye ratio are calculated. And the size and proportion of the text content in the candidate video frame are calculated. If all the above preset conditions are met, the candidate video frame is used as the target video frame, and then the target video frame is analyzed. Specifically, the depth score is calculated and the depth score size is compared. If the depth score also meets the requirements, the loop continues to find other target video frames. If a new target video frame cannot be extracted, the end video frame will be extracted, and the previously obtained target video frame will be analyzed and processed. This includes determining whether there is text content. If so, it will be erased. If not, it will determine whether to draw the background. If necessary, the background will be drawn. If not, the property text information will be directly filled in, and the text and border will be drawn. After the drawing is completed, it will be added to the cover, and the desired cover will be obtained.

[0146] Figure 6 The structure diagram of an electronic device provided by an exemplary embodiment of the present application is shown in FIG. Figure 6 As shown, it includes: a memory 60a and a processor 60b; wherein the memory 60a is used to store a computer program; the processor 60b is coupled to the memory 60a, and is used to execute the computer program to perform the following steps:

[0147] The processor 60b is used to obtain at least one candidate video frame containing a house image and a person image in the house video to be processed;

[0148] Extracting human body contour information and facial status information of a person in the person image of the at least one candidate video frame;

[0149] Using the human body contour information to determine whether a person in the at least one candidate video frame appears within a legal area; and

[0150] Using the facial state information, determining whether the facial state information of the person in the at least one candidate video frame meets a state condition;

[0151] In response to the judgment result that the person in the candidate video frame appears in the legal area and the facial state information of the person meets the state condition, extracting the candidate video frame from the house video to be processed as the target video frame;

[0152] The extracted target video frame is added to the cover position of the house video to be processed.

[0153] The processor 60b is used to identify the house image contained in the candidate video and mark a prohibited display area in the candidate video frame;

[0154] Determine the coordinate range corresponding to the prohibited display area;

[0155] According to the human body contour information, determining whether a key point of the human body contour appears within the coordinate range of the prohibited display area;

[0156] If the judgment result is that the human body contour key point does not appear within the coordinate range of the prohibited display area, multiple human body contour key points in the human body contour information are used to judge whether the display range of the person image in the alternative video frame at least includes the upper body range of the person.

[0157] The processor 60b is used to determine the human body contour coordinates corresponding to the multiple contour key points in the human body contour information;

[0158] If the number of the key points of the outline falling into the prohibited display area is greater than the number threshold according to the human body outline coordinates, and / or if the maximum distance of the edge of the key points of the outline falling into the prohibited display area is greater than the distance threshold according to the outline coordinates, then the person image is displayed in the prohibited display area;

[0159] If it is determined based on the contour coordinates that the number of contour key points falling into the prohibited display area is not greater than the number threshold, and the maximum distance between the contour key points falling into the prohibited display area and the edge is not greater than the distance threshold, then the person image is not displayed in the prohibited display area.

[0160] The processor 60b is used to determine the facial key points constituting the facial contour in the facial state information and the corresponding facial coordinate range; determine whether the area ratio of the facial coordinate range in the candidate video meets the first area condition;

[0161] Determine the eye key points constituting the eye contour in the facial information and the corresponding eye coordinate range; and determine whether the area ratio of the eye coordinate range in the facial coordinate range meets the second area condition.

[0162] Optionally, the multiple contour key points contained in the human body contour information include at least one of a top of the head key point, a shoulder key point, an upper limb key point, an upper torso key point, and a lower torso key point;

[0163] The processor 60b is used to determine the key points of the outline representing the person based on the human body outline information; determine whether the key points of the outline include the key points of the top of the head, the key points of the shoulders, the key points of the upper limbs, and the key points of the upper torso representing the upper body, and do not include the key points of the lower torso representing the lower body of the person;

[0164] If so, the display range of the person image in the alternative video frame is the upper body range of the person.

[0165] The processor 60b is used for obtaining depth information corresponding to the image of the person in the candidate video frame in response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition;

[0166] Using the depth information, determining whether the distance relationship between the person and the camera shooting the video of the house meets the distance condition;

[0167] If it meets the requirements, the candidate video frame is determined as the target video frame.

[0168] The processor 60b is used to determine whether the target video frame contains text content;

[0169] If there is text content, the real estate information content is drawn on the cover after erasing the text content according to the text generation rules; wherein the text generation rules include: at least one of text size, text content and text effect; or,

[0170] If there is text content, the real estate information content is overlaid on the text content according to the display specifications of the text content; wherein the display specifications include: at least one of display area size, display area coordinates and display transparency;

[0171] The target video frame containing the real estate information content is added to the cover position of the house video to be processed.

[0172] The processor 60b is configured to score the candidate video frame using a pre-trained scene scoring model if no candidate video frame containing a person image is found;

[0173] According to the scoring value, a frame is selected from the candidate video frames as the cover.

[0174] It should be noted that the specific implementation of the above modules or units provided in this embodiment can refer to the above Figure 1 The relevant description of the illustrated method embodiment will not be repeated here.

[0175] An exemplary embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes one or more processors to execute the steps in the above-mentioned electronic contract processing method. The detailed implementation process of each step can be found in the relevant description of the above-mentioned embodiments, which will not be repeated here.

[0176] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0177] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0178] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0180] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0181] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0182] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transit media), such as modulated data signals and carrier waves.

[0183] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0184] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A cover generation method, characterized in that: Applied to the server side, the method includes: Acquire at least one candidate video frame containing a house image and a person image in the house video to be processed; Extracting human body contour information and facial status information of a person in the person image of the at least one candidate video frame; Using the human body contour information to determine whether a person in the at least one candidate video frame appears within a legal area; and Using the facial state information, determining whether the facial state information of the person in the at least one candidate video frame meets a state condition; In response to the judgment result that the person in the candidate video frame appears in the legal area and the facial state information of the person meets the state condition, extracting the candidate video frame from the house video to be processed as the target video frame; The extracted target video frame is added to the cover position of the house video to be processed.

2. The method according to claim 1, characterized in that The step of using the human body contour information to determine whether a person in the at least one candidate video frame appears within a legal area includes: Identify the house image contained in the candidate video, and mark a prohibited display area in the candidate video frame; Determine the coordinate range corresponding to the prohibited display area; According to the human body contour information, determining whether a key point of the human body contour appears within the coordinate range of the prohibited display area; If the judgment result is that the human body contour key point does not appear within the coordinate range of the prohibited display area, multiple human body contour key points in the human body contour information are used to judge whether the display range of the person image in the alternative video frame at least includes the upper body range of the person.

3. The method according to claim 2, characterized in that The step of determining whether a plurality of key points of the human body contour information appear within the coordinate range of the prohibited display area comprises: Determine the human body contour coordinates corresponding to a plurality of contour key points in the human body contour information; If the number of the key points of the outline falling into the prohibited display area is greater than the number threshold according to the human body outline coordinates, and / or if the maximum distance of the edge of the key points of the outline falling into the prohibited display area is greater than the distance threshold according to the outline coordinates, then the person image is displayed in the prohibited display area; If it is determined based on the contour coordinates that the number of contour key points falling into the prohibited display area is not greater than the number threshold, and the maximum distance between the contour key points falling into the prohibited display area and the edge is not greater than the distance threshold, then the person image is not displayed in the prohibited display area.

4. The method according to claim 2, characterized in that: The step of using the facial state information to determine whether the facial state information of the person in the at least one candidate video frame meets the state condition includes: Determine the facial key points constituting the facial contour in the facial state information and the corresponding facial coordinate range; determine whether the area ratio of the facial coordinate range in the candidate video meets the first area condition; Determine the eye key points constituting the eye contour in the facial information and the corresponding eye coordinate range; and determine whether the area ratio of the eye coordinate range in the facial coordinate range meets the second area condition.

5. The method according to claim 2, characterized in that: A plurality of contour key points included in the human body contour information, wherein the plurality of contour key points include at least one of a top of the head key point, a shoulder key point, an upper limb key point, an upper torso key point, and a lower torso key point; The step of using the multiple contour key points in the human body contour information to determine whether the display range of the person image in the candidate video frame at least includes the upper body range of the person includes: Determine key points of the human body contour based on the human body contour information; Determine whether the outline key points include the top of the head key points, the shoulder key points, the upper limb key points, and the upper torso key points representing the upper body, and do not include the lower torso key points representing the lower body of the person; If so, the display range of the person image in the alternative video frame is the upper body range of the person.

6. The method according to claim 2, characterized in that: In response to the person appearing in the legal area in the candidate video frame and the judgment result that the person meets the status condition, extracting the candidate video frame from the house video to be processed as the target video frame includes: In response to a judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition, acquiring depth information corresponding to the person image in the candidate video frame; Using the depth information, determining whether the distance relationship between the person and the camera shooting the video of the house meets the distance condition; If it meets the requirements, the candidate video frame is determined as the target video frame.

7. The method according to claim 2, characterized in that The step of adding the extracted target video frame to the cover position of the house video to be processed comprises: Determining whether the target video frame contains text content; If there is text content, the real estate information content is drawn on the cover after erasing the text content according to the text generation rules; wherein the text generation rules include: at least one of text size, text content and text effect; or, If there is text content, the real estate information content is overlaid on the text content according to the display specifications of the text content; wherein the display specifications include: at least one of display area size, display area coordinates and display transparency; The target video frame containing the real estate information content is added to the cover position of the house video to be processed.

8. The method according to claim 1, characterized in that Also includes: If no candidate video frame containing a person image is found, the candidate video frame is scored using a pre-trained scene scoring model; According to the scoring value, a frame is selected from the candidate video frames as the cover.

9. A cover generation device, characterized in that: The device comprises: An acquisition module, used for acquiring at least one candidate video frame containing a house image and a person image in the house video to be processed; An extraction module, used to extract the human body contour information and facial state information of the person in the person image of the at least one candidate video frame; A judgment module, used to judge whether a person in the at least one candidate video frame appears in a legal area by using the human body contour information; and The judging module is further configured to use the facial status information to judge whether the person in the at least one candidate video frame meets the status condition; The extraction module is further configured to extract the candidate video frame as the target video frame from the house video to be processed in response to the judgment result that the person in the candidate video frame appears in the legal area and the person meets the status condition; The adding module is used to add the extracted target video frame to the cover position of the house video to be processed.

10. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to store a computer program; and the processor is coupled to the memory and is used to execute the computer program to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 8.

Images