A video information processing method and system based on AIGC

Abstract

The present invention relates to the field of video processing technology, and in particular to a video information processing method and system based on AIGC, comprising processing an image acquired by a customer soliciting robot, dividing a person foreground and an environment background in the image, extracting various motion features in the environment background, and determining the occlusion priority of each motion feature; generating a plurality of intermediate frame images, combining them to form a background video; determining an offset evaluation parameter; determining whether the operation of the background video is qualified based on an environment background overlap parameter and an offset evaluation parameter; determining whether the training for the generation of the background video is completed, and using the trained video generation parameters to process the input image acquired by the customer soliciting robot to generate a video; verifying whether the generated background video is verified based on a test image and a verification image, and when it is determined that the background video is abnormal, adjusting the generation parameters of the background video, thereby improving the production efficiency of the video.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video information processing method and system based on AIGC. Background Art

[0002] AIGC, or artificial intelligence generated content, is a field that applies artificial intelligence technology to content creation. With the continuous development of deep learning and generative models, AIGC has made significant progress in many fields, from image generation to music composition, to text writing and video generation, showing great potential. The technology of AIGC technology can greatly reduce the difficulty of many tasks, and video production is one of them. Now a system is needed to process the image information obtained by the customer-attracting robot based on the AIGC technology to improve the efficiency of video production.

[0003] Chinese Patent Publication No.: CN117135416B discloses a video generation system based on AIGC, including an information input module, a text parsing module, a model selection module and a video generation module, wherein the information input module is used to input user demand information, the text parsing module is used to perform content analysis on the demand information, the model selection module is used to select a basic model of the video, and the video generation module is used to process the basic model to obtain a complete video; the system can intelligently identify the text information input by the user, select a suitable model from the model library, and virtually shoot the processing process of the model to obtain the required video; it can be seen that the above technical solution has the following problems: it does not consider targeted processing of the background, does not consider training the video generation module through sample videos before actual video generation, and cannot adjust the generation parameters of the video generation module according to the generated background video, which affects the production efficiency of the video. Summary of the invention

[0004] To this end, the present invention provides a video information processing method and system based on AIGC, so as to overcome the problem that the prior art does not consider targeted processing of the background, does not consider training the video generation module with a sample video before actual video generation, and cannot adjust the generation parameters of the video generation module according to the generated background video, resulting in low video production efficiency.

[0005] On the one hand, the present invention provides a video information processing method based on AIGC, comprising:

[0006] S1, dividing the foreground of the person and the background of the environment in the image based on the first frame image and the last frame image of the intercepted sample video, extracting various motion features in the background of the environment, and determining the occlusion priority of each motion feature based on the first frame image and the last frame image;

[0007] S2, generating a plurality of intermediate frame images based on the motion features and occlusion priorities in the first frame image and the last frame image, and combining the intermediate frame images to form a background video;

[0008] S3, acquiring a test image based on the background video, acquiring a verification image based on the sample video, determining an environmental background overlap parameter based on the test image and the verification image, determining a feature offset parameter of each motion feature based on the test image and the verification image, and determining an offset evaluation parameter based on each feature offset parameter;

[0009] S4, determining whether the background video is qualified based on the environment background overlap parameter and the offset evaluation parameter, including:

[0010] Determine that the background video runs properly, and execute step S5.

[0011] Or, determine that the background video is running abnormally, obtain each abnormal area, determine the abnormal area ratio based on each abnormal area, and determine the processing method for the background video based on the abnormal area ratio, wherein the processing method is to adjust the brightness change speed of each generated intermediate frame image to a corresponding value, or to adjust the number of intermediate frame images generated for a single abnormal area to a corresponding value;

[0012] S5, when it is determined that the background video runs properly, repeating steps S1-S4 until the generation of background videos corresponding to all sample videos is completed;

[0013] S6, determining that the training of background video generation is completed, and using the trained video generation parameters to process the input image acquired by the customer soliciting robot to generate a video, wherein the video generation parameters include the brightness change speed of each intermediate frame image and the number of intermediate frame images.

[0014] Furthermore, in S1, the process of determining the occlusion priority of each motion feature based on the first frame image and the last frame image includes:

[0015] Extract various motion features in the background environment;

[0016] Calculate the priority evaluation parameters of each motion feature;

[0017] For a single motion feature, the ratio of its area in the last frame image to its area in the first frame image is calculated to obtain the area change parameter;

[0018] Overlap the last frame image with the first frame image, obtain the position distance of a single motion feature on the overlap plane, solve the ratio of the position distance to the width of the first frame image, and obtain the displacement change parameter;

[0019] The area change parameter and the displacement change parameter are respectively assigned corresponding weight coefficients and summed to obtain the priority evaluation parameter for a single motion feature;

[0020] The weight coefficient corresponding to the area change parameter is the area weight coefficient, and the weight coefficient corresponding to the displacement change parameter is the displacement weight coefficient;

[0021] Compare the priority evaluation parameters, which are proportional to the occlusion priority.

[0022] Furthermore, in S3, the process of determining the environmental background overlap parameter based on the inspection image and the verification image includes:

[0023] Extracting the environmental background in the verification image, and determining the extracted environmental background as the verification background image;

[0024] The features of each object in the inspection image and the background image to be checked are compared for overlap, and the ratio of the total area of ​​the overlapped regions of the features of each object to the total area of ​​the features of each object is obtained to obtain the overlap parameter of the background environment;

[0025] In S4, the process of determining whether the background video is qualified based on the environmental background overlap parameter includes:

[0026] If the environment background overlap parameter is less than or equal to the first preset background overlap parameter, it is determined that the background video is abnormal, and a processing method for the background video is determined based on the abnormal area ratio;

[0027] If the environment background overlap parameter is less than or equal to the second preset background overlap parameter and greater than the first preset background overlap parameter, determining whether the operation of the background video is qualified based on the offset evaluation parameter;

[0028] If the environment background overlap parameter is greater than the second preset background overlap parameter, it is determined that the background video is running properly.

[0029] Furthermore, in S4, the process of determining whether the running of the background video is qualified based on the offset evaluation parameter includes:

[0030] Obtain feature offset parameters of a single running feature, overlap and compare the inspection image and the background verification image, obtain the distance between the single motion feature and the background verification image, and obtain feature offset parameters for the single running feature;

[0031] Solve the offset average value of each feature offset parameter, calculate the ratio of the offset average value to the width of the test image, and obtain the offset evaluation parameter;

[0032] If the offset evaluation parameter is less than or equal to the preset offset evaluation parameter, the background video is judged to be running qualified;

[0033] If the offset evaluation parameter is greater than a preset offset evaluation parameter, the displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter.

[0034] Further, the displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter, wherein:

[0035] The increase of the displacement weight coefficient determined based on the offset evaluation parameter is proportional to the offset evaluation parameter.

[0036] Furthermore, in S4, the process of obtaining each abnormal area and determining a processing method for the background video based on the abnormal area ratio includes:

[0037] Obtain the abnormal area, compare the inspection image and the background image, and determine the area corresponding to the motion features that do not match the comparison as the abnormal area.

[0038] For a single abnormal region, determining a ratio of an area of ​​the abnormal region to an area of ​​a corresponding motion feature as an area ratio;

[0039] Solve the average value of each area ratio to get the abnormal area ratio;

[0040] If the abnormal area ratio is greater than the preset abnormal area ratio, a regional distribution parameter is determined based on each abnormal area, and a processing method for the background video is determined based on the regional distribution parameter;

[0041] If the abnormal area ratio is less than or equal to the preset abnormal area ratio, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

[0042] Furthermore, in S4, the process of determining the regional distribution parameter based on each abnormal region includes:

[0043] Obtain the distance between each abnormal area, and solve the variance of each distance to obtain the regional distribution parameter;

[0044] The process of determining a processing method for the background video based on the regional distribution parameters includes:

[0045] If the regional distribution parameter is less than or equal to the preset regional distribution parameter, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter;

[0046] If the regional distribution parameter is greater than the preset regional distribution parameter, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

[0047] Further, in S6, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter, wherein:

[0048] The increase in the number of generated intermediate frame images determined based on the regional distribution parameter is inversely proportional to the regional distribution parameter.

[0049] Further, in S6, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio, wherein:

[0050] The reduction in the brightness change rate determined based on the abnormal area ratio is proportional to the abnormal area ratio.

[0051] On the other hand, the present invention also provides a video information processing system using the above-mentioned video information processing method, comprising:

[0052] A sample storage module, which includes a number of sample videos for training;

[0053] A sample processing module, which is connected to the sample input module and is used to capture the first frame image and the last frame image of a single sample video;

[0054] A video generation module, which is connected to the sample processing module and is used to generate a plurality of intermediate frame images based on the first frame image and the last frame image, the video generation module includes a data division unit for dividing the foreground of the person and the background of the environment in the image, a feature extraction unit for extracting each motion feature in the background of the environment, and an occlusion control unit for determining the occlusion priority of each motion feature based on the first frame image and the last frame image;

[0055] A video fitting module, connected to the video generation module, for combining the intermediate frame images to form a background video;

[0056] A training module, which is connected to the video fitting module and the video generation module, is used to verify the background video and determine the operation of the background video, wherein:

[0057] In the case of verifying the background video, a test image is obtained based on the background video, a verification image is obtained based on the sample video, an environmental background overlap parameter is determined based on the test image and the verification image, a feature offset parameter of each motion feature is determined one by one based on the test image and the verification image, an offset evaluation parameter is determined based on each feature offset parameter, and whether the operation of the background video is qualified is determined based on the environmental background overlap parameter and the offset evaluation parameter, wherein,

[0058] The video generation module is determined as a reliable module when the background video is qualified.

[0059] Or, when it is determined that the background video is running abnormally, each abnormal area is obtained, and a processing method for the background video is determined based on the abnormal area ratio, including:

[0060] Adjust the brightness change speed of each intermediate frame image to the corresponding value.

[0061] Or, the number of intermediate frame images generated for a single abnormal region is adjusted to a corresponding value.

[0062] Compared with the prior art, the beneficial effect of the present invention lies in that various motion features in the environmental background are obtained to carry out targeted processing of the environmental background, and before actual video generation, the video generation module is trained with sample videos, and the generated background video is verified based on the test image and the verification image, and when it is determined that the background video is abnormal, the generation parameters of the video generation module are adjusted, thereby improving the video production efficiency.

[0063] Furthermore, the occlusion priority of each motion feature is determined by calculating the priority evaluation parameters of each motion feature. The area change parameter represents the change in the size of the object over time. The closer the object is to the camera in the scene, the larger the object is. The greater the magnitude of the object's increase over time, the closer it is to the camera, and thus the higher its occlusion priority. The displacement change parameter represents the moving speed of the object. An object with a faster moving speed and moving toward the camera is more likely to occlude other objects and thus has a higher occlusion priority. The priority evaluation parameter represents the movement law of the object. By calculating the priority evaluation parameter as the data basis for determining the priority of each operation feature, the accuracy of video generation is improved while further improving the efficiency of video generation.

[0064] Furthermore, the operation of the background video is determined to be qualified by the environment background overlap parameter, and the generated background video test image is extracted and compared with the sample video to determine whether the generated video is qualified, and further determine whether the operation of the background video is qualified. The environment background overlap parameter characterizes the conformity of the generated single-frame test image with the verification image of the sample video. When the environment background overlap parameter is less than or equal to the first preset background overlap parameter, there is a large deviation between the test image and the verification background image as a whole, and the operation of the background video is determined to be abnormal, and the specific processing method for the corresponding abnormal situation is determined; when the environment background overlap parameter is less than or equal to the second preset background overlap parameter and greater than the first preset background overlap parameter, the offset evaluation parameter is further determined, and the offset The evaluation parameter characterizes the error of each motion feature. When the offset evaluation parameter is less than or equal to the preset offset evaluation parameter, although there is a positional offset in each motion feature, the overall offset is small on average. In this case, the background video is judged to be qualified. When the offset evaluation parameter is greater than the preset offset evaluation parameter, although the overall deviation between the inspection image and the background image is small, the positional deviation of some motion features is too large. At this time, due to the misjudgment of occlusion, the motion trajectory of the motion feature is predicted, and the positional deviation of the motion feature is incorrectly predicted, resulting in excessive positional deviation. At this time, the priority evaluation parameter is adjusted to adjust the data determination model determined by the priority according to the actual video generation situation, thereby further improving the accuracy of video generation.

[0065] Furthermore, when determining that the background video is abnormal in operation, the abnormal area ratio is determined, and the processing method for the background video is determined based on the abnormal area ratio. The abnormal area represents the area corresponding to the motion feature that generates the abnormality in the comparison results of the test image and the verification background image. The area ratio represents the overlap of the motion features corresponding to the abnormal area, and the abnormal area ratio represents the comprehensive situation of each abnormal area; when it is determined that the abnormal area ratio is less than or equal to the preset abnormal area ratio, in this case the environmental background overlap parameter is less than or equal to the first preset background overlap parameter, and there is a large deviation between the test image and the verification background image as a whole, but the average offset range of each motion feature determined is small. At this time, due to the abnormal pixel brightness, there are a large number of abnormal areas that affect the determination of the overall offset range of each motion feature. In this case, the brightness change speed of each intermediate frame image is corrected, and the brightness of each pixel point of the video is accurately determined to improve the video continuity; in the abnormal area When the proportion is greater than the preset abnormal area proportion, the regional distribution parameter is determined based on each abnormal area, and the processing method for the background video is determined based on the regional distribution parameter. The regional distribution parameter characterizes the distribution of the abnormal area. When the regional distribution parameter is less than or equal to the preset regional distribution parameter, the abnormal area is concentrated. In this case, the video is discontinuous due to the low number of intermediate frame images generated, so that the motion characteristics of position deviation appear concentratedly. At this time, the number of intermediate frame videos is increased to make the changes between adjacent frames smoother, reducing sudden visual jumps and unnatural transitions, thereby effectively reducing the appearance of abnormal areas; when the regional distribution parameter is greater than the preset regional distribution parameter, the abnormal areas appear in a scattered manner. At this time, due to the large jump in the change of each pixel point, the pixel brightness of each scattered reflective point in the video screen deviates. At this time, the brightness change speed is adjusted to ensure the coherent generation of the video while further improving the generation efficiency of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of a video information processing method based on AIGC according to an embodiment of the present invention;

[0067] Figure 2 A module block diagram of a video information processing system based on AIGC according to an embodiment of the present invention;

[0068] Figure 3 A logic determination diagram for determining whether the background video is qualified based on the environmental background overlap parameter according to an embodiment of the present invention;

[0069] Figure 4 This is a logic decision diagram for determining whether the operation of a background video is qualified based on an offset evaluation parameter according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0071] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0072] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0073] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0074] See also Figure 1 As shown, they are respectively flowcharts of the steps of a video information processing method based on AIGC according to an embodiment of the present invention. An embodiment of the present invention provides a video information processing method based on AIGC, including:

[0075] S1, dividing the foreground of the person and the background of the environment in the image based on the first frame image and the last frame image of the intercepted sample video, extracting various motion features in the background of the environment, and determining the occlusion priority of each motion feature based on the first frame image and the last frame image;

[0076] S2, generating a plurality of intermediate frame images based on the motion features and occlusion priorities in the first frame image and the last frame image, and combining the intermediate frame images to form a background video;

[0077] S3, acquiring a test image based on the background video, acquiring a verification image based on the sample video, determining an environmental background overlap parameter based on the test image and the verification image, determining a feature offset parameter of each motion feature based on the test image and the verification image, and determining an offset evaluation parameter based on each feature offset parameter;

[0078] S4, determining whether the background video is qualified based on the environment background overlap parameter and the offset evaluation parameter, including:

[0079] Determine that the background video runs properly, and execute step S5.

[0080] Or, determine that the background video is running abnormally, obtain each abnormal area, determine the abnormal area ratio based on each abnormal area, and determine the processing method for the background video based on the abnormal area ratio, wherein the processing method is to adjust the brightness change speed of each generated intermediate frame image to a corresponding value, or to adjust the number of intermediate frame images generated for a single abnormal area to a corresponding value;

[0081] S5, when it is determined that the background video runs properly, repeating steps S1-S4 until the generation of background videos corresponding to all sample videos is completed;

[0082] S6, determining that the training of background video generation is completed, and using the trained video generation parameters to process the input image acquired by the customer soliciting robot to generate a video, wherein the video generation parameters include the brightness change speed of each intermediate frame image and the number of intermediate frame images.

[0083] Specifically, there is no limitation on the specific method by which the data division unit divides the foreground of characters and the background of the environment in the image. The features of each character in the image can be extracted through an algorithm, and each character feature can be determined as the foreground of the character, and the area in the image except for the image of each character can be determined as the background of the environment. This will not be repeated.

[0084] Specifically, there is no limitation on the specific method by which the feature extraction unit extracts the motion features in the environmental background. The edges of objects in the environmental background can be detected by algorithms such as the Sobel operator and the Canny edge detection. The edges outline the contours of the objects. The contours of the single object and the area within the contours of the object are determined as single object features. The positions of the features of the objects in the first frame image and the last frame image are compared in turn, and the features of the objects that are displaced are determined as motion features. This will not be repeated.

[0085] Specifically, there is no limitation on the specific method of obtaining the test image based on the background video. The video frame corresponding to the middle time point of the background video can be obtained as the test image, or any frame of the video can be used as the test image. The verification image is the video frame in the sample video corresponding to the same time node as the test image. This will not be repeated.

[0086] Specifically, the generation of videos often results in large errors due to the complex background. This solution improves the accuracy of video generation by performing targeted training on the generation of environmental backgrounds.

[0087] Specifically, the motion features in the environmental background are obtained to perform targeted processing on the environmental background. Before actual video generation, the video generation module is trained with sample videos. The generated background video is verified based on the test image and the verification image. When it is determined that the background video is abnormal, the generation parameters of the video generation module are adjusted, thereby improving the video production efficiency.

[0088] See also Figure 2 As shown, they are module block diagrams of a video information processing system based on AIGC according to an embodiment of the present invention. The video information processing system based on AIGC according to the present invention comprises:

[0089] A sample storage module, which includes a number of sample videos for training;

[0090] A sample processing module, which is connected to the sample input module and is used to capture the first frame image and the last frame image of a single sample video;

[0091] A video generation module, which is connected to the sample processing module and is used to generate a plurality of intermediate frame images based on the first frame image and the last frame image, the video generation module includes a data division unit for dividing the foreground of the person and the background of the environment in the image, a feature extraction unit for extracting each motion feature in the background of the environment, and an occlusion control unit for determining the occlusion priority of each motion feature based on the first frame image and the last frame image;

[0092] A video fitting module, connected to the video generation module, for combining the intermediate frame images to form a background video;

[0093] A training module, which is connected to the video fitting module and the video generation module, is used to verify the background video and determine the operation of the background video, wherein:

[0094] When verifying the background video, a test image is obtained based on the background video, a verification image is obtained based on the sample video, an environmental background overlap parameter is determined based on the test image and the verification image, a feature offset parameter of each motion feature is determined one by one based on the test image and the verification image, an offset evaluation parameter is determined based on each feature offset parameter, and whether the operation of the background video is qualified is determined based on the environmental background overlap parameter and the offset evaluation parameter, wherein,

[0095] The video generation module is determined as a reliable module when the background video is qualified.

[0096] Or, when it is determined that the background video is running abnormally, each abnormal area is obtained, and a processing method for the background video is determined based on the abnormal area ratio, including:

[0097] Adjust the brightness change speed of each intermediate frame image to the corresponding value.

[0098] Or, the number of intermediate frame images generated for a single abnormal region is adjusted to a corresponding value.

[0099] See also Figure 3 As shown, they are respectively logic determination diagrams for determining whether the background video is qualified based on the environmental background overlap parameter in an embodiment of the present invention. In S1 of the present invention, the process of determining the occlusion priority of each motion feature based on the first frame image and the last frame image includes:

[0100] Extract various motion features in the background environment;

[0101] Calculate the priority evaluation parameters of each motion feature;

[0102] For a single motion feature, the ratio of its area in the last frame image to its area in the first frame image is calculated to obtain the area change parameter;

[0103] Overlap the last frame image with the first frame image, obtain the position distance of a single motion feature on the overlap plane, solve the ratio of the position distance to the width of the first frame image, and obtain the displacement change parameter;

[0104] The area change parameter and the displacement change parameter are respectively assigned corresponding weight coefficients and summed to obtain the priority evaluation parameter for a single motion feature;

[0105] The weight coefficient corresponding to the area change parameter is the area weight coefficient, and the weight coefficient corresponding to the displacement change parameter is the displacement weight coefficient;

[0106] Compare the priority evaluation parameters, which are proportional to the occlusion priority.

[0107] Specifically, the area weight coefficient is 0.72 and the displacement weight coefficient is 0.28.

[0108] Specifically, the occlusion priority of each motion feature is determined by calculating the priority evaluation parameters of each motion feature. The area change parameter represents the change in the size of the object over time. The closer the object is to the camera in the scene, the larger the object is. The greater the magnitude of the object's increase over time, the closer it is to the camera, and thus the higher its occlusion priority. The displacement change parameter represents the moving speed of the object. An object with a faster moving speed and moving toward the camera is more likely to occlude other objects and thus has a higher occlusion priority. The priority evaluation parameter represents the movement law of the object. By calculating the priority evaluation parameter as the data basis for determining the priority of each operation feature, the accuracy of video generation is improved while further improving the efficiency of video generation.

[0109] See also Figure 4As shown, they are respectively logic decision diagrams for determining whether the operation of the background video is qualified based on the offset evaluation parameter in an embodiment of the present invention. In S3 of the present invention, the process of determining the environment background overlap parameter based on the inspection image and the verification image includes:

[0110] Extracting the environmental background in the verification image, and determining the extracted environmental background as the verification background image;

[0111] The features of each object in the inspection image and the background image to be checked are compared for overlap, and the ratio of the total area of ​​the overlapped regions of the features of each object to the total area of ​​the features of each object is obtained to obtain the overlap parameter of the background environment;

[0112] In S4, the process of determining whether the background video is qualified based on the environmental background overlap parameter includes:

[0113] If the environment background overlap parameter is less than or equal to the first preset background overlap parameter, it is determined that the background video is abnormal, and a processing method for the background video is determined based on the abnormal area ratio;

[0114] If the environment background overlap parameter is less than or equal to the second preset background overlap parameter and greater than the first preset background overlap parameter, determining whether the operation of the background video is qualified based on the offset evaluation parameter;

[0115] If the environment background overlap parameter is greater than the second preset background overlap parameter, it is determined that the background video is running properly.

[0116] Specifically, the first preset background overlap parameter is selected within the interval [0.85, 0.88], and the second preset background overlap parameter is selected within the interval [0.96, 0.98].

[0117] Specifically, the process of determining whether the operation of the background video is qualified based on the offset evaluation parameter includes:

[0118] Obtain feature offset parameters of a single running feature, overlap and compare the inspection image and the background verification image, obtain the distance between the single motion feature and the background verification image, and obtain feature offset parameters for the single running feature;

[0119] Solve the offset average value of each feature offset parameter, calculate the ratio of the offset average value to the width of the test image, and obtain the offset evaluation parameter;

[0120] If the offset evaluation parameter is less than or equal to the preset offset evaluation parameter, the background video is judged to be running qualified;

[0121] If the offset evaluation parameter is greater than a preset offset evaluation parameter, the displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter.

[0122] Specifically, the preset offset evaluation parameter P0 is selected within the interval [0.03, 0.08].

[0123] Specifically, the operation of the background video is determined to be qualified through the environment background overlap parameter, and the generated background video test image is extracted and compared with the sample video to determine whether the generated video is qualified, and further determine whether the operation of the background video is qualified. The environment background overlap parameter characterizes the conformity of the generated single-frame test image with the verification image of the sample video. When the environment background overlap parameter is less than or equal to the first preset background overlap parameter, there is a large deviation between the test image and the verification background image as a whole, and the operation of the background video is determined to be abnormal, and the specific processing method for the corresponding abnormal situation is determined; when the environment background overlap parameter is less than or equal to the second preset background overlap parameter and greater than the first preset background overlap parameter, the offset evaluation parameter is further determined, and the offset The evaluation parameter characterizes the error of each motion feature. When the offset evaluation parameter is less than or equal to the preset offset evaluation parameter, although there is a positional offset in each motion feature, the overall offset is small on average. In this case, the background video is judged to be qualified. When the offset evaluation parameter is greater than the preset offset evaluation parameter, although the overall deviation between the inspection image and the background image is small, the positional deviation of some motion features is too large. At this time, due to the misjudgment of occlusion, the motion trajectory of the motion feature is predicted, and the positional deviation of the motion feature is incorrectly predicted, resulting in excessive positional deviation. At this time, the priority evaluation parameter is adjusted to adjust the data determination model determined by the priority according to the actual video generation situation, thereby further improving the accuracy of video generation.

[0124] Specifically, the displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter, wherein:

[0125] The increase of the displacement weight coefficient determined based on the offset evaluation parameter is proportional to the offset evaluation parameter.

[0126] In this embodiment, optionally,

[0127] Comparing the offset evaluation parameter with a first preset evaluation comparison threshold and a second preset evaluation comparison threshold;

[0128] If the offset evaluation parameter is less than or equal to the first preset evaluation comparison threshold, the displacement weight coefficient is adjusted to 1.11 times the initial displacement weight coefficient;

[0129] If the offset evaluation parameter is less than or equal to the second preset evaluation comparison threshold and greater than the first preset evaluation comparison threshold, the displacement weight coefficient is adjusted to 1.21 times the initial displacement weight coefficient;

[0130] If the offset evaluation parameter is greater than the second preset evaluation comparison threshold, the displacement weight coefficient is adjusted to 1.29 times the initial displacement weight coefficient;

[0131] The first preset evaluation comparison threshold is 2.2P0, and the second preset evaluation comparison threshold is 3.2P0.

[0132] Specifically, in S4, the process of obtaining each abnormal area and determining a processing method for the background video based on the abnormal area ratio includes:

[0133] Obtain the abnormal area, compare the inspection image and the background image, and determine the area corresponding to the motion features that do not match the comparison as the abnormal area.

[0134] For a single abnormal region, determining a ratio of an area of ​​the abnormal region to an area of ​​a corresponding motion feature as an area ratio;

[0135] Solve the average value of each area ratio to get the abnormal area ratio;

[0136] If the abnormal area ratio is greater than the preset abnormal area ratio, a regional distribution parameter is determined based on each abnormal area, and a processing method for the background video is determined based on the regional distribution parameter;

[0137] If the abnormal area ratio is less than or equal to the preset abnormal area ratio, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

[0138] Specifically, the preset abnormal area ratio B0 is selected within the interval [1.11, 1.38].

[0139] Specifically, the brightness change speed of each intermediate frame image is the ratio of the brightness difference between the corresponding pixels of the two intermediate frames to the value of the interval frame number between the two intermediate frames increased by one, which will not be described in detail.

[0140] Specifically, it can be understood that in order to ensure the quality of the generated image, the comparison method for images or features during the video generation training process is not only the comparison of contour shapes, but also includes the comparison of the brightness of each pixel in the image area to calculate the difference in color value and brightness value of the corresponding pixel to determine whether the pixels overlap. If there is a difference, it will be determined that the pixels do not overlap. This will not be repeated.

[0141] Specifically, the process of determining regional distribution parameters based on each abnormal area includes:

[0142] Obtain the distance between each abnormal area, and solve the variance of each distance to obtain the regional distribution parameter;

[0143] The process of determining a processing method for the background video based on the regional distribution parameters includes:

[0144] If the regional distribution parameter is less than or equal to the preset regional distribution parameter, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter;

[0145] If the regional distribution parameter is greater than the preset regional distribution parameter, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

[0146] Specifically, the preset regional distribution parameter F0 is selected within the interval [0.64C0, 0.81C0], and C0 is the width of the first frame image.

[0147] Specifically, when determining that the background video is running abnormally, the abnormal area ratio is determined, and the processing method for the background video is determined based on the abnormal area ratio. The abnormal area represents the area corresponding to the motion feature that generates the abnormality in the comparison results of the test image and the verification background image. The area ratio represents the overlap of the motion features corresponding to the abnormal area, and the abnormal area ratio represents the comprehensive situation of each abnormal area; when it is determined that the abnormal area ratio is less than or equal to the preset abnormal area ratio, in this case the environmental background overlap parameter is less than or equal to the first preset background overlap parameter, and there is a large deviation between the test image and the verification background image as a whole, but the average offset range of each motion feature determined is small. At this time, due to the abnormal pixel brightness, there are a large number of abnormal areas that affect the determination of the overall offset range of each motion feature. In this case, the brightness change speed of each intermediate frame image is corrected, and the brightness of each pixel point in the video is accurately determined to improve the video continuity; in the abnormal area When the proportion is greater than the preset abnormal area proportion, the regional distribution parameter is determined based on each abnormal area, and the processing method for the background video is determined based on the regional distribution parameter. The regional distribution parameter characterizes the distribution of the abnormal area. When the regional distribution parameter is less than or equal to the preset regional distribution parameter, the abnormal area is concentrated. In this case, the video is discontinuous due to the low number of intermediate frame images generated, so that the motion characteristics of position deviation appear concentratedly. At this time, the number of intermediate frame videos is increased to make the changes between adjacent frames smoother, reducing sudden visual jumps and unnatural transitions, thereby effectively reducing the appearance of abnormal areas; when the regional distribution parameter is greater than the preset regional distribution parameter, the abnormal areas appear in a scattered manner. At this time, due to the large jump in the change of each pixel point, the pixel brightness of each scattered reflective point in the video screen deviates. At this time, the brightness change speed is adjusted to ensure the coherent generation of the video while further improving the generation efficiency of the video.

[0148] Specifically, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter, where:

[0149] The increase in the number of generated intermediate frame images determined based on the regional distribution parameter is inversely proportional to the regional distribution parameter.

[0150] In this embodiment, optionally,

[0151] Comparing the regional distribution parameter with a first preset regional comparison threshold and a second preset regional comparison threshold;

[0152] If the regional distribution parameter is less than or equal to the first preset regional comparison threshold, the number of generated intermediate frame images is adjusted to 1.32 times the initial number of generated images;

[0153] If the regional distribution parameter is less than or equal to the second preset regional comparison threshold and greater than the first preset regional comparison threshold, the number of generated intermediate frame images is adjusted to 1.22 times the initial number of generated images;

[0154] If the regional distribution parameter is greater than the second preset regional comparison threshold, the number of generated intermediate frame images is adjusted to 1.11 times the initial number of generated images;

[0155] The first preset region comparison threshold is 0.82F0, and the second preset region comparison threshold is 0.51F0.

[0156] Specifically, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio, where:

[0157] The reduction in the brightness change rate determined based on the abnormal area ratio is proportional to the abnormal area ratio.

[0158] In this embodiment, optionally,

[0159] Compare the abnormal area ratio with a first preset ratio comparison threshold and a second preset ratio comparison threshold;

[0160] If the abnormal area ratio is less than or equal to the first preset ratio comparison threshold, the brightness change speed is adjusted to 0.93 times the initial brightness change speed;

[0161] If the abnormal area ratio is less than or equal to the second preset ratio comparison threshold and greater than the first preset ratio comparison threshold, the brightness change speed is adjusted to 0.87 times the initial brightness change speed;

[0162] If the abnormal area ratio is greater than the second preset ratio comparison threshold, the brightness change speed is adjusted to 0.81 times the initial brightness change speed;

[0163] The first preset ratio comparison threshold is 0.77B0, and the second preset ratio comparison threshold is 0.98B0.

[0164] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video information processing method based on AIGC, characterized in that: include: S1, dividing the foreground of the person and the background of the environment in the image based on the first frame image and the last frame image of the intercepted sample video, extracting various motion features in the background of the environment, and determining the occlusion priority of each motion feature based on the first frame image and the last frame image; S2, generating a plurality of intermediate frame images based on the motion features and occlusion priorities in the first frame image and the last frame image, and combining the intermediate frame images to form a background video; S3, acquiring a test image based on the background video, acquiring a verification image based on the sample video, determining an environmental background overlap parameter based on the test image and the verification image, determining a feature offset parameter of each motion feature based on the test image and the verification image, and determining an offset evaluation parameter based on each feature offset parameter; S4, determining whether the background video is qualified based on the environment background overlap parameter and the offset evaluation parameter, including: Determine that the background video runs properly, and execute step S5. Or, determine that the background video is running abnormally, obtain each abnormal area, determine the abnormal area ratio based on each abnormal area, and determine the processing method for the background video based on the abnormal area ratio, wherein the processing method is to adjust the brightness change speed of each generated intermediate frame image to a corresponding value, or to adjust the number of intermediate frame images generated for a single abnormal area to a corresponding value; S5, when it is determined that the background video runs properly, repeating steps S1-S4 until the generation of background videos corresponding to all sample videos is completed; S6, determining that the training of background video generation is completed, and processing the images acquired by the customer soliciting robot based on the trained video generation parameters to generate a video, wherein the video generation parameters include the brightness change speed of each intermediate frame image and the number of intermediate frame images.

2. The video information processing method based on AIGC according to claim 1, characterized in that: In S1, the process of determining the occlusion priority of each motion feature based on the first frame image and the last frame image includes: Extract various motion features in the background environment; Calculate the priority evaluation parameters of each motion feature; For a single motion feature, the ratio of its area in the last frame image to its area in the first frame image is calculated to obtain the area change parameter; Overlap the last frame image with the first frame image, obtain the position distance of a single motion feature on the overlap plane, solve the ratio of the position distance to the width of the first frame image, and obtain the displacement change parameter; The area change parameter and the displacement change parameter are respectively assigned corresponding weight coefficients and summed to obtain the priority evaluation parameter for a single motion feature; The weight coefficient corresponding to the area change parameter is the area weight coefficient, and the weight coefficient corresponding to the displacement change parameter is the displacement weight coefficient; Compare the priority evaluation parameters, which are proportional to the occlusion priority.

3. The video information processing method based on AIGC according to claim 2, characterized in that: In S3, the process of determining the environment background overlap parameter based on the inspection image and the verification image includes: Extracting the environmental background in the verification image, and determining the extracted environmental background as the verification background image; The features of each object in the inspection image and the background image to be checked are compared for overlap, and the ratio of the total area of ​​the overlapped regions of the features of each object to the total area of ​​the features of each object is obtained to obtain the overlap parameter of the background environment; In S4, the process of determining whether the background video is qualified based on the environmental background overlap parameter includes: If the environment background overlap parameter is less than or equal to the first preset background overlap parameter, it is determined that the background video is abnormal, and a processing method for the background video is determined based on the abnormal area ratio; If the environment background overlap parameter is less than or equal to the second preset background overlap parameter and greater than the first preset background overlap parameter, determining whether the operation of the background video is qualified based on the offset evaluation parameter; If the environment background overlap parameter is greater than the second preset background overlap parameter, it is determined that the background video is running properly.

4. The video information processing method based on AIGC according to claim 3 is characterized in that: In S4, the process of determining whether the running of the background video is qualified based on the offset evaluation parameter includes: Obtaining feature offset parameters of a single motion feature, overlaying and comparing the inspection image and the background verification image, obtaining the distance between the inspection image and the background verification image, and obtaining feature offset parameters for the single motion feature; Solve the offset average value of each feature offset parameter, calculate the ratio of the offset average value to the width of the test image, and obtain the offset evaluation parameter; If the offset evaluation parameter is less than or equal to the preset offset evaluation parameter, the background video is judged to be running qualified; If the offset evaluation parameter is greater than a preset offset evaluation parameter, the displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter.

5. The video information processing method based on AIGC according to claim 4 is characterized in that: The displacement weight coefficient is adjusted to a corresponding value based on the offset evaluation parameter, wherein: The increase of the displacement weight coefficient determined based on the offset evaluation parameter is proportional to the offset evaluation parameter.

6. The video information processing method based on AIGC according to claim 5, characterized in that: In S4, the process of obtaining each abnormal area and determining a processing method for the background video based on the abnormal area ratio includes: Obtain the abnormal area, compare the inspection image and the background image, and determine the area corresponding to the motion features that do not match the comparison as the abnormal area. For a single abnormal region, determining a ratio of an area of ​​the abnormal region to an area of ​​a corresponding motion feature as an area ratio; Solve the average value of each area ratio to get the abnormal area ratio; If the abnormal area ratio is greater than the preset abnormal area ratio, a regional distribution parameter is determined based on each abnormal area, and a processing method for the background video is determined based on the regional distribution parameter; If the abnormal area ratio is less than or equal to the preset abnormal area ratio, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

7. The video information processing method based on AIGC according to claim 6, characterized in that: In S4, the process of determining the regional distribution parameters based on each abnormal region includes: Obtain the distance between each abnormal area, and solve the variance of each distance to obtain the regional distribution parameter; The process of determining a processing method for the background video based on the regional distribution parameters includes: If the regional distribution parameter is less than or equal to the preset regional distribution parameter, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter; If the regional distribution parameter is greater than the preset regional distribution parameter, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio.

8. The video information processing method based on AIGC according to claim 7, characterized in that: In S6, the number of generated intermediate frame images is adjusted to a corresponding value based on the regional distribution parameter, wherein: The increase in the number of generated intermediate frame images determined based on the regional distribution parameter is inversely proportional to the regional distribution parameter.

9. The video information processing method based on AIGC according to claim 8, characterized in that: In S6, the brightness change speed of each generated intermediate frame image is adjusted to a corresponding value based on the abnormal area ratio, wherein: The reduction in the brightness change rate determined based on the abnormal area ratio is proportional to the abnormal area ratio.

10. A video information processing system using the video information processing method according to any one of claims 1 to 9, characterized in that: include, A sample storage module, which is used to store a number of sample videos for training; A sample processing module, which is connected to the sample storage module and is used to capture the first frame image and the last frame image of a single sample video; A video generation module, which is connected to the sample processing module and is used to generate a plurality of intermediate frame images based on the first frame image and the last frame image, the video generation module includes a data division unit for dividing the foreground of the person and the background of the environment in the image, a feature extraction unit for extracting each motion feature in the background of the environment, and an occlusion control unit for determining the occlusion priority of each motion feature based on the first frame image and the last frame image; A video fitting module, which is connected to the video generation module and is used to combine the intermediate frame images to form a background video; A training module, which is connected to the video fitting module and the video generation module, is used to verify the background video and determine the operation of the background video, wherein: In the case of verifying the background video, a test image is obtained based on the background video, a verification image is obtained based on the sample video, an environmental background overlap parameter is determined based on the test image and the verification image, a feature offset parameter of each motion feature is determined one by one based on the test image and the verification image, an offset evaluation parameter is determined based on each feature offset parameter, and whether the operation of the background video is qualified is determined based on the environmental background overlap parameter and the offset evaluation parameter, wherein, The video generation module is determined as a reliable module when the background video is qualified. Or, when it is determined that the background video is running abnormally, each abnormal area is obtained, and a processing method for the background video is determined based on the abnormal area ratio, including: Adjust the brightness change speed of each intermediate frame image to the corresponding value. Or, the number of intermediate frame images generated for a single abnormal region is adjusted to a corresponding value.

Images