Motion speed information generation method and device and related equipment

Abstract

The embodiment of the invention provides a motion speed information generation method and device and related equipment, and relates to the technical field of video, and the method comprises the steps: determining an image set based on an obtained video file, the image set comprises a plurality of images arranged according to a time sequence, the time intervals between adjacent images are the same, and each image comprises N pixel points; according to the motion positions of the N pixel points in the adjacent images and the time interval between the adjacent images, the motion speed of the N pixel points is calculated, an optical flow field corresponding to the image set is obtained, and the optical flow field is used for representing a vector field of motion of the pixel points in the multiple sets of adjacent images; according to the first optical flow value of the optical flow field, a target speed grade is determined in a plurality of preset speed grades, each preset speed grade corresponds to a preset second optical flow value, and the target speed grade is used for representing the movement speed of the video file. The accuracy of the generated motion speed information is improved, so that the subsequent decision based on the motion speed is more efficient and accurate.

Description

Technical Field

[0001] The present invention relates to the field of video technology, and in particular to a motion speed information generation method, device and related equipment. Background Art

[0002] Dynamic analysis of video content is an important task in the current field of computer vision and video processing. For example, in application scenarios such as autonomous driving, monitoring systems, and motion analysis, the movement speed of objects on the road can be determined through dynamic analysis of the video, and further appropriate autonomous driving strategies can be generated based on the movement speed of the objects, thereby improving the safety of autonomous driving and the accuracy of the strategy.

[0003] In the existing video dynamic analysis process, the determination of motion speed is generally based on the subjective perception of the operator. However, different operators may perceive the same video at different speeds, and the same operator may perceive different speeds at different videos. Even the same operator may perceive the same video at different times. This results in the generated motion speed information being less accurate, which in turn affects subsequent decisions based on motion speed. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device and related equipment for generating motion speed information to solve the problem of poor accuracy in determining motion speed in the existing video dynamic analysis process. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a method for generating motion speed information is provided, the method comprising:

[0006] Based on the acquired video file, an image set is determined, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer;

[0007] Calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images;

[0008] According to the first optical flow value of the optical flow field, a target speed grade is determined among multiple preset speed grades, each of the preset speed grades corresponds to a preset second optical flow value, and the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the movement speed of the video file.

[0009] Optionally, calculating the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain the optical flow field corresponding to the image set includes:

[0010] According to the moving positions between N pixels in the first image and N pixels in the second image, and the time interval between the first image and the second image, the moving speed of the N pixels along the target direction is calculated to obtain the moving speed of the N pixels in a plurality of groups of adjacent images along the target direction, wherein the first image and the second image are any groups of adjacent images in the image set;

[0011] The movement speeds of N pixels in the plurality of groups of adjacent images along the target direction are combined to obtain an optical flow field corresponding to the image set.

[0012] Optionally, determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field comprises:

[0013] When the direction of the first optical flow value of the optical flow field is the same as the direction of the preset second optical flow value, respectively calculating the optical flow value differences between the first optical flow value of the optical flow field and the second optical flow values ​​corresponding to a plurality of preset speed classifications, to obtain a plurality of optical flow value differences;

[0014] The preset speed classification corresponding to the minimum optical flow value difference among the multiple optical flow value differences is determined as the target speed classification.

[0015] Optionally, after determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field, the method further comprises:

[0016] After associating the target speed classification with the optical flow field corresponding to the image set, the target device is used to output a generated video, the generated video includes multiple groups of generated adjacent images, the vector fields of pixel point motion in the multiple groups of generated adjacent images match the optical flow field corresponding to the image set, and the preset format includes at least one of tabular data, image data and text data.

[0017] Optionally, before determining a target speed classification among a plurality of preset speed classifications according to the first optical flow value of the optical flow field, the method further comprises:

[0018] Performing feature recognition on at least two images in the image set to determine feature information of the video file;

[0019] The plurality of preset speed levels are acquired according to the characteristic information of the video file, wherein the characteristic information of the object described by each preset speed level in the plurality of preset speed levels matches the characteristic information of the video file.

[0020] Optionally, before obtaining the video file based on the image set, the method further includes:

[0021] Reading continuous frames of the acquired video file at a preset time interval to obtain frame data;

[0022] The frame data is preprocessed to obtain the image set, wherein the preprocessing includes standardization of image size and image color.

[0023] In a second aspect of the present invention, a motion speed information generating device is provided, the device comprising:

[0024] A first acquisition module is used to determine an image set based on the acquired video file, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer;

[0025] A calculation module, used to calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images, so as to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images;

[0026] A generating module is used to determine a target speed grade among a plurality of preset speed grades according to the first optical flow value of the optical flow field, each of the preset speed grades corresponds to a preset second optical flow value, the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the motion speed of the video file.

[0027] In a third aspect of the present invention, there is provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0028] Memory, used to store programs;

[0029] The processor is used to implement the method described in the first aspect when executing the program stored in the memory.

[0030] In a fourth aspect of the implementation of the present invention, a readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0031] In a fifth aspect of the implementation of the present invention, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the method described in the first aspect.

[0032] In an embodiment of the present invention, in the process of determining an image set, images can be captured at the same time interval in chronological order, so that the time difference between adjacent images is constant, and the motion speed can be directly associated with the position change, thereby improving the accuracy and efficiency of motion speed calculation; based on adjacent images in the image set, the motion speed of each pixel is calculated to obtain an optical flow field corresponding to the image set, and the motion speed information is enriched by describing the motion of each pixel in the image set at adjacent time points, so that more complex motion patterns can be captured; and the motion speed is preset and graded, and the motion speed information of the image set is generated according to the relationship between the first optical flow value in the optical flow field and the second optical flow value preset in the preset speed classification, which helps to understand the dynamic changes of the video content, improves the accuracy of the generated motion speed information, and makes subsequent decisions based on motion speed more efficient and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0034] Figure 1 This is a flow chart of a method for generating motion speed information according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the structure of a server in an embodiment of the present invention;

[0036] Figure 3 FIG2 is a second flow chart of a method for generating motion speed information according to an embodiment of the present invention;

[0037] Figure 4 Schematic diagram of the structure of a motion speed information generating device in an embodiment of the present invention;

[0038] Figure 5 It is a structural schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0041] In addition, the terms "first", "second", etc. may be used in this article to describe various directions, actions, steps or elements, but these directions, actions, steps or elements are not limited by these terms. These terms are only used to distinguish a first direction, action, step or element from another direction, action, step or element. The terms "first", "second", etc. cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0042] The embodiment of the present invention provides a method for generating motion speed information, such as Figure 1 As shown, the steps of the method include:

[0043] Step 101: determining an image set based on an acquired video file, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer;

[0044] The video file may be a video data stream captured by a camera. For example, it may be a video file obtained by a camera at a fixed position recording a preset scene (for example, a surveillance video of a highway captured by a surveillance camera set on a highway), and then the video file is frame-intercepted in time sequence to obtain an image set; it may also be a video file obtained by a mobile camera recording a specific scene (for example, a video of a short-distance race in a playground captured by a sports camera set on a playground track), and then the video file is frame-intercepted in time sequence to obtain an image set; it may also be a video file obtained by a mobile camera recording a non-specific scene (for example, a video of a vehicle driving process captured by a visual camera set on the roof), and then the video file is frame-intercepted in time sequence to obtain an image set. In the process of intercepting the acquired video file, images can be intercepted at the same time interval. In this way, the obtained image set includes multiple images arranged in time sequence, and the time intervals between adjacent images in the multiple images are the same. In the process of video dynamic analysis, what is actually being done is the motion analysis of the pixels in the image. By capturing images at the same time interval, when calculating the motion speed of N pixels in the image, the motion speed can be directly associated with the position change, thereby improving the accuracy and efficiency of subsequent calculations.

[0045] Step 102: Calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain an optical flow field corresponding to the image set, where the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images.

[0046] Optical flow is a two-dimensional vector field used to describe the motion of pixels in an image sequence. In computer vision, it represents the position change of each pixel between two or more consecutive time points. The motion speed of each pixel can be decomposed into horizontal and vertical components. In this example, the horizontal speed is determined as the motion speed of N pixels, and the optical flow algorithm can be used to calculate the motion of each pixel between adjacent frames. For example, convolutional neural networks (CNN) are used to estimate the optical flow to obtain the motion speed (motion vector) of each pixel, and then the motion vectors of all pixels are merged to generate an optical flow field. The optical flow field can describe the motion of each pixel in the video frame sequence at adjacent time points, showing the dynamic changes of the entire picture. In this way, the motion speed of N pixels is calculated based on the motion positions of N pixels in adjacent images and the time interval between adjacent images, enriching the motion speed information, and capturing more complex motion patterns (for example, uniform motion followed by accelerated motion, deceleration motion followed by uniform motion, acceleration motion followed by deceleration motion, and deceleration motion followed by acceleration motion, etc.). This facilitates the subsequent generation of videos that are more consistent with text descriptions (such as acceleration, deceleration, etc.) in application scenarios of video generation (such as application scenarios of Wensheng videos).

[0047] Step 103: determine a target speed grade from a plurality of preset speed grades according to the first optical flow value of the optical flow field, each of the preset speed grades corresponds to a preset second optical flow value, the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the movement speed of the video file.

[0048] For example, the degree of motion of each pixel is measured by calculating the size of the optical flow value (i.e., amplitude). The size of the optical flow value reflects the distance moved by the pixel within a preset time. Then, a set of thresholds (i.e., the preset second optical flow values) are set according to the application scenario and data characteristics to distinguish different types of motion. These thresholds will determine how to divide the preset speed grades. For example:

[0049] Still: The optical flow value is equal to 0, and the corresponding movement speed can be V1, V1 is equal to 0;

[0050] Slow motion: The optical flow value is less than a small threshold (such as 1), and the corresponding motion speed can be V2;

[0051] Medium-speed movement: The optical flow value is between two medium thresholds (e.g. 1 to 5), and the corresponding movement speed can be V3;

[0052] Fast movement: The optical flow value is greater than a larger threshold (such as 5), and the corresponding movement speed may be V4.

[0053] In other words, the preset speed classification may include stillness, slow motion, medium speed motion, and fast motion.

[0054] It should be understood that the preset speed levels of slow motion, medium-speed motion and fast motion may give different sensory experiences in different video scenes. For example, in a video of pedestrians moving between buildings, the motion speed V2 corresponding to slow motion may be 0km / h to 1km / h, and the motion speed V4 corresponding to fast motion may be greater than 5km / h; in a video of a vehicle driving on a road, the motion speed V2 corresponding to slow motion may be 0km / h to 10km / h, and the motion speed V4 corresponding to fast motion may be greater than 80km / h. Therefore, different preset speed levels can be set for different video scenes, and each preset speed level corresponds to a preset second optical flow value and a corresponding preset speed range. The second optical flow value can also be adjusted according to actual conditions.

[0055] In addition, more preset speed classifications can be set to further improve accuracy. For example, medium-speed motion can be subdivided into: first medium-speed motion, second medium-speed motion, third medium-speed motion and fourth medium-speed motion. The preset threshold (i.e., the second optical flow value) corresponding to the first medium-speed motion can be 1 to 2, the preset threshold corresponding to the second medium-speed motion can be 2 to 3, the preset threshold corresponding to the third medium-speed motion can be 3 to 4, and the preset threshold corresponding to the fourth medium-speed motion can be 4 to 5. The same technical effect can be achieved, which will not be repeated here.

[0056] Wherein, the optical flow value and the movement speed are both vectors, and in the process of determining the target speed classification, the optical flow direction needs to correspond to the movement speed direction. In this example, the component in the horizontal direction is used as an example for exemplary description, and it should be understood that the component in the vertical direction is also applicable.

[0057] First, decompose according to the direction of the optical flow in the optical flow field to obtain the first optical flow value in the horizontal direction of the optical flow field; then compare the relationship between the first optical flow value in the horizontal direction and the corresponding threshold value in the preset speed classification (i.e., the preset second optical flow value) to determine the target speed classification in multiple preset speed classifications, thereby determining the horizontal speed of the video file. For example, if the first optical flow value in the horizontal direction is equal to 0, the preset speed classification of the image set in the horizontal direction is still, and the corresponding movement speed can be V1; if the first optical flow value in the horizontal direction is equal to 0.4, the preset speed classification of the image set in the horizontal direction is slow movement, and the corresponding movement speed can be V2; if the first optical flow value in the horizontal direction is equal to 3, the preset speed classification of the image set in the horizontal direction is medium speed movement, and the corresponding movement speed can be V3; if the first optical flow value in the horizontal direction is equal to 6, the preset speed classification of the image set in the horizontal direction is fast movement, and the corresponding movement speed can be V4. Classifying the motion speed of each pixel in the image calculated based on optical flow into stillness, slow motion, medium-speed motion, and fast motion helps to understand the dynamic changes of video content, improves the accuracy of determining motion speed, and makes subsequent decisions based on motion speed more efficient and accurate.

[0058] In an embodiment of the present invention, in the process of determining an image set, images can be captured at the same time interval in chronological order, so that the time difference between adjacent images is constant, and the motion speed can be directly associated with the position change, thereby improving the accuracy and efficiency of motion speed calculation; based on adjacent images in the image set, the motion speed of each pixel is calculated to obtain an optical flow field corresponding to the image set, and by describing the motion of each pixel in the image set at adjacent time points, the motion speed information is enriched, and more complex motion patterns can be captured; and the motion speed is preset and graded, and the motion speed information of the image set is generated according to the relationship between the first optical flow value in the optical flow field and the second optical flow value preset in the preset speed classification, which helps to understand the dynamic changes of the video content, improves the accuracy of the generated motion speed information, and makes subsequent decisions based on motion speed more efficient and accurate.

[0059] Optionally, before the step 101 of obtaining the video file and determining the image set, the method further includes:

[0060] Reading continuous frames of the acquired video file at a preset time interval to obtain frame data;

[0061] The frame data is preprocessed to obtain the image set, wherein the preprocessing includes standardization of image size and image color.

[0062] In one example, the preset time interval may be 1 second. The acquired video file is read at a frequency of 1 frame per second to capture multiple frames of images (i.e., frame data) arranged in time sequence; then, the frame data is preprocessed to obtain an image set. The preprocessing may be a standardization process, which may include standardization of image size, standardization of image color, and noise removal. Through the standardization process, a more stable and reliable data foundation can be established, thereby improving the overall accuracy of optical flow calculation.

[0063] The preprocessing of the frame data may include: using an image scaling method to standardize the image size of the frame data so that each image in the image set has a uniform resolution, that is, has the same size. In this way, when the motion speed of each pixel is calculated according to adjacent images in the image set in the subsequent step, the situation in which the output calculation result has a large error due to the inconsistency of the size of the input image is reduced.

[0064] The preprocessing of the frame data may also include: normalizing the pixel values ​​to the range of [0, 1] or [-1, 1] to perform image color standardization on the frame data. For example, the original red, green, blue (RGB) values ​​may be normalized by dividing them by 255. This helps to reduce the differences under different brightness conditions and improve the robustness of subsequent optical flow calculations.

[0065] The preprocessing of the frame data may also include: removing noise from the frame data by using a smoothing filtering technique such as Gaussian blur, so as to improve the final motion estimation accuracy.

[0066] Optionally, the step 102 of calculating the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain the optical flow field corresponding to the image set includes:

[0067] According to the moving positions between N pixels in the first image and N pixels in the second image, and the time interval between the first image and the second image, the moving speed of the N pixels along the target direction is calculated to obtain the moving speed of the N pixels in a plurality of groups of adjacent images along the target direction, wherein the first image and the second image are any groups of adjacent images in the image set;

[0068] The movement speeds of N pixels in the plurality of groups of adjacent images along the target direction are combined to obtain an optical flow field corresponding to the image set.

[0069] In this example, taking any adjacent first image and second image in the image set as an example, the preset time interval between capturing the first image and capturing the second image can be 1 second. In addition, each image in the image set is a standardized image, so the first image and the second image have the same size, that is, the number of pixels in the first image is the same as the number of pixels in the second image. Based on the optical flow algorithm, the movement speed of the N pixels along the target direction can be determined according to the movement position between the N pixels in the first image and the N pixels in the second image, and the time interval between the first image and the second image. Among them, the target direction can be a horizontal direction. In some examples, the movement direction of the pixel point can be a non-horizontal direction. At this time, the movement direction of the pixel point can be decomposed, and the component in the horizontal direction is determined as the movement speed along the target direction.

[0070] According to the speed of each pixel along the target direction, the position change of the pixel in the first image and the second image between two consecutive time points can be determined; similarly, the position change of the pixel in each image in the image set between two consecutive time points can be obtained, and then the movement speed of N pixels in multiple groups of adjacent images along the target direction can be combined to determine the optical flow field corresponding to the image set. The optical flow field can describe the movement of each pixel in the video frame sequence at adjacent time points, showing the dynamic changes of the entire picture. In this way, the motion speed information is enriched, and more complex motion patterns can be captured (for example, uniform motion followed by accelerated motion, decelerated motion followed by uniform motion, accelerated motion followed by decelerated motion, and decelerated motion followed by accelerated motion, etc.). It can effectively solve the problems of low precision, sensitivity to noise, and difficulty in handling complex scenes existing in traditional methods, and has broad application prospects.

[0071] Optionally, the step 103, determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field, comprises:

[0072] When the direction of the first optical flow value of the optical flow field is the same as the direction of the preset second optical flow value, respectively calculating the optical flow value differences between the first optical flow value of the optical flow field and the second optical flow values ​​corresponding to a plurality of preset speed classifications, to obtain a plurality of optical flow value differences;

[0073] The preset speed classification corresponding to the minimum optical flow value difference among the multiple optical flow value differences is determined as the target speed classification.

[0074] In this example, the preset speed classification may include stillness, slow motion, medium speed motion, and fast motion. Among them, the second optical flow value corresponding to stillness may be 0; the second optical flow value corresponding to slow motion may be 0 to 1; the second optical flow value corresponding to medium speed motion may be 1 to 5; and the second optical flow value corresponding to fast motion may be greater than 5. Decompose according to the direction of the optical flow in the optical flow field to obtain the first optical flow value in the horizontal direction; then compare the first optical flow value with the second optical flow value in the preset speed classification to determine the target speed classification of the image set in the horizontal direction.

[0075] If the first optical flow value in the horizontal direction is equal to 0, and the optical flow value difference between the second optical flow value 0 corresponding to stillness in the preset speed classification and the first optical flow value 0 of the current optical flow field has a minimum value of 0, then it can be determined that the target speed classification of the image set in the horizontal direction is stillness, and the corresponding motion speed can be V1;

[0076] If the first optical flow value in the horizontal direction is equal to 0.4, the second optical flow value corresponding to slow motion in the preset speed classification includes 0.4, so the optical flow value difference between the second optical flow value corresponding to slow motion and the first optical flow value 0.4 of the current optical flow field has a minimum value of 0, then it can be determined that the target speed classification of the image set in the horizontal direction is slow motion, and the corresponding motion speed can be V2;

[0077] If the first optical flow value in the horizontal direction is equal to 3, the second optical flow value corresponding to the medium-speed motion in the preset speed classification includes 3, and the optical flow value difference between the second optical flow value corresponding to the medium-speed motion includes 3 and the first optical flow value 3 of the current optical flow field has a minimum value of 0, then it can be determined that the target speed classification of the image set in the horizontal direction is medium-speed motion, and the corresponding motion speed can be V3;

[0078] If the first optical flow value in the horizontal direction is equal to 6, the second optical flow value corresponding to rapid motion in the preset speed classification includes 6, and therefore the optical flow value difference between the second optical flow value corresponding to rapid motion includes 6 and the first optical flow value 6 of the current optical flow field has a minimum value of 0, then it can be determined that the target speed classification of the image set in the horizontal direction is rapid motion, and the corresponding motion speed can be V4.

[0079] In this way, according to the first optical flow value in the optical flow field and the second optical flow value preset in the preset speed classification, the target speed classification of the image set is determined in the preset speed classification, and the motion speed of each pixel in the image calculated based on the optical flow is classified into stillness, slow motion, medium speed motion and fast motion, and the speed classification is performed according to the size and direction of the optical flow, so as to achieve more accurate and efficient video motion analysis, help understand the dynamic changes of the video content, improve the accuracy of the generated motion speed information, and make subsequent decisions based on motion speed more efficient and accurate. It is convenient to generate videos that are more in line with text descriptions (such as stillness, slow motion, medium speed motion and fast motion, etc.) in subsequent application scenarios of generating videos (such as application scenarios of Wensheng videos).

[0080] Optionally, after determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field in step 103, the method further comprises:

[0081] After associating the target speed classification with the optical flow field corresponding to the image set, the target device is used to output a generated video, the generated video includes multiple groups of generated adjacent images, the vector fields of pixel point motion in the multiple groups of generated adjacent images match the optical flow field corresponding to the image set, and the preset format includes at least one of tabular data, image data and text data.

[0082] The embodiment of the present invention provides a method for generating motion speed information, which can be executed by a server, such as Figure 2 As shown, the server may include a video input module, an optical flow estimation module, a speed classification module, and an output module. Figure 3 As shown, a method for generating motion speed information includes the following steps:

[0083] First, the server can obtain video files from the video acquisition device. The video input module can read continuous frames of the obtained video files at a preset time interval, and then standardize the frame data to obtain an image set, which can establish a more stable and reliable data foundation, thereby improving the overall accuracy of optical flow calculation. The optical flow estimation module can use optical flow to calculate the motion speed of each pixel, and then obtain the optical flow field corresponding to the image set according to the motion speed of each pixel. The optical flow field can describe the motion of each pixel in the video frame sequence at adjacent time points, showing the dynamic changes of the entire picture, enriching the motion speed information, and capturing more complex motion patterns. The speed classification module can perform motion classification based on the speed size and direction, and measure the degree of motion of each pixel by calculating the size of the light value (i.e., amplitude). The size of the optical flow value reflects the moving distance of the pixel within the preset time, and then set a set of thresholds according to the application scenario and data characteristics. These thresholds will determine how to divide the preset speed classification, which helps to understand the dynamic changes of the video content, improve the accuracy of the generated motion speed information, and make subsequent decisions based on motion speed more efficient and accurate. The output module can output the motion speed information to the target device in a preset format, and the preset format includes at least one of table data, image data and text data. In this way, the motion speed information of each pixel can be output comprehensively, and a variety of data formats suitable for different application scenarios can be generated. These formats include clearly structured table data, intuitive image data and detailed text data descriptions, which are convenient for subsequent target devices to generate videos that are more in line with the text description during the video generation process. It is of great significance for the data processing, analysis and visualization of subsequent target devices.

[0084] In one example, for each pixel, a table containing its position (x, y), horizontal displacement, vertical displacement, motion amplitude and direction can be created. This table can be exported in a structured format such as a CSV file for subsequent processing or analysis.

[0085] In another example, the motion speed information of each pixel can also be visualized to make the changes and motion patterns clear at a glance. This output usually uses color coding to represent different motion characteristics. First, it can be converted into the HSV color space according to the amplitude and direction of the motion, where the hue represents the direction and the brightness represents the amplitude; then the HSV image can be converted into a standard RGB image for easy display and storage.

[0086] In another example, a text summary report of the overall scene may also be provided, which may include global average motion speed, maximum and minimum speeds, main motion directions, etc. Outputting these statistical information in the form of structured text facilitates subsequent dynamic analysis of the video content.

[0087] Optionally, before determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field in step 103, the method further comprises:

[0088] Performing feature recognition on at least two images in the image set to determine feature information of the video file;

[0089] The plurality of preset speed levels are acquired according to the characteristic information of the video file, wherein the characteristic information of the object described by each preset speed level in the plurality of preset speed levels matches the characteristic information of the video file.

[0090] In this example, considering the preset speed levels of slow motion, medium speed motion and fast motion, the sensory experience may be different in different video scenes. Based on this, multiple groups of different preset speed levels can be set for different video scenes. For example, for the video scene of pedestrian movement, 4 preset speed levels are set; for the video scene of vehicle movement, another 4 preset speed levels are set. Among them, the preset speed corresponding to the same preset speed level in different video scenes can be different. For example, the slow motion in the video scene of pedestrian movement can correspond to a motion speed of 0km / h to 1km / h; and the slow motion in the video scene of vehicle movement can correspond to a motion speed of 0km / h to 10km / h. Among them, the threshold (i.e., the second optical flow value) corresponding to the same preset speed level in different video scenes can also be different. For example, the slow motion in the video scene of pedestrian movement can correspond to a second optical flow value of 0.5; and the slow motion in the video scene of vehicle movement can correspond to a second optical flow value of 1. Different preset speed levels can be set for different video scenes, and each preset speed level corresponds to a preset second optical flow value and a corresponding preset speed range, which can be adjusted according to actual conditions.

[0091] Before determining the target speed classification among multiple preset speed classifications, first, feature recognition is performed on at least two images in the image set to determine the feature information of the video file. Specifically, computer vision technology can be used to extract image features, such as edge detection (Canny, Sobel, etc.) or pre-trained convolutional neural networks, etc., to perform feature recognition on any two images in the image set to determine the feature information of the video file. Then, according to the feature information of the video file, multiple preset speed classifications matching the feature information of the video file are obtained. For example, if the feature information represents that the image set is a video scene of pedestrian movement, then in the subsequent process of determining the target speed classification, multiple preset speed classifications corresponding to the video scene of pedestrian movement can be selected; if the feature information represents that the image set is a video scene of vehicle movement, then in the subsequent process of determining the target speed classification, multiple preset speed classifications corresponding to the video scene of vehicle movement can be selected. In this way, the matching of the preset speed classification and the video scene is improved, and when the target speed classification is determined among multiple preset speed classifications according to the first optical flow value of the optical flow field, the accuracy is further improved, which helps to understand the dynamic changes of the video content, making the subsequent decision based on the motion speed more efficient and accurate. In subsequent application scenarios of generating videos (such as application scenarios of Wensheng videos), videos that are more consistent with the text description can be generated for specific video scenes.

[0092] like Figure 4 As shown, the embodiment of the present invention further provides a motion speed information generating device 400, comprising:

[0093] A first acquisition module 401 is used to determine an image set based on the acquired video file, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer;

[0094] A calculation module 402 is used to calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images, so as to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images;

[0095] A generation module 403 is used to determine a target speed grade from multiple preset speed grades according to the first optical flow value of the optical flow field, each of the preset speed grades corresponds to a preset second optical flow value, and the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the movement speed of the video file.

[0096] Optionally, the calculation module 402 is specifically configured to:

[0097] According to the moving positions between N pixels in the first image and N pixels in the second image, and the time interval between the first image and the second image, the moving speed of the N pixels along the target direction is calculated to obtain the moving speed of the N pixels in a plurality of groups of adjacent images along the target direction, wherein the first image and the second image are any groups of adjacent images in the image set;

[0098] The movement speeds of N pixels in the plurality of groups of adjacent images along the target direction are combined to obtain an optical flow field corresponding to the image set.

[0099] Optionally, the generating module 403 is specifically configured to:

[0100] When the direction of the first optical flow value of the optical flow field is the same as the direction of the preset second optical flow value, respectively calculating the optical flow value differences between the first optical flow value of the optical flow field and the second optical flow values ​​corresponding to a plurality of preset speed classifications, to obtain a plurality of optical flow value differences;

[0101] The preset speed classification corresponding to the minimum optical flow value difference among the multiple optical flow value differences is determined as the target speed classification.

[0102] Optionally, the motion speed information generating device 400 further includes:

[0103] An output module is used to associate the target speed classification with the optical flow field corresponding to the image set and then output it to a target device in a preset format. The target device is used to output a generated video. The generated video includes multiple groups of generated adjacent images. The vector fields of pixel point motion in the multiple groups of generated adjacent images match the optical flow field corresponding to the image set. The preset format includes at least one of tabular data, image data and text data.

[0104] Optionally, the motion speed information generating device 400 further includes:

[0105] A recognition module, used to perform feature recognition on at least two images in the image set to determine feature information of the video file;

[0106] The second acquisition module is used to acquire the multiple preset speed levels according to the characteristic information of the video file, wherein the characteristic information of the object described by each preset speed level in the multiple preset speed levels matches the characteristic information of the video file.

[0107] Optionally, the motion speed information generating device 400 further includes:

[0108] A reading module is used to read continuous frames of the acquired video file at a preset time interval to obtain frame data;

[0109] The preprocessing module is used to preprocess the frame data to obtain the image set, wherein the preprocessing includes standardization of image size and image color.

[0110] The motion speed information generating device 400 provided by the embodiment of the present invention can achieve Figure 1 The various processes implemented by the method embodiment shown can achieve the same beneficial effects, and will not be described again here to avoid repetition.

[0111] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0112] The memory 503 is used to store computer programs. The processor 501 is used to execute the programs stored in the memory 503 to implement the following steps:

[0113] Based on the acquired video file, an image set is determined, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer;

[0114] Calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images;

[0115] According to the first optical flow value of the optical flow field, a target speed grade is determined among multiple preset speed grades, each of the preset speed grades corresponds to a preset second optical flow value, and the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the movement speed of the video file.

[0116] Optionally, calculating the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain the optical flow field corresponding to the image set includes:

[0117] According to the moving positions between N pixels in the first image and N pixels in the second image, and the time interval between the first image and the second image, the moving speed of the N pixels along the target direction is calculated to obtain the moving speed of the N pixels in a plurality of groups of adjacent images along the target direction, wherein the first image and the second image are any groups of adjacent images in the image set;

[0118] The movement speeds of N pixels in the plurality of groups of adjacent images along the target direction are combined to obtain an optical flow field corresponding to the image set.

[0119] Optionally, determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field comprises:

[0120] When the direction of the first optical flow value of the optical flow field is the same as the direction of the preset second optical flow value, respectively calculating the optical flow value differences between the first optical flow value of the optical flow field and the second optical flow values ​​corresponding to a plurality of preset speed classifications, to obtain a plurality of optical flow value differences;

[0121] The preset speed classification corresponding to the minimum optical flow value difference among the multiple optical flow value differences is determined as the target speed classification.

[0122] Optionally, the processor 501 is further configured to implement the following steps when executing a program stored in the memory 503:

[0123] After associating the target speed classification with the optical flow field corresponding to the image set, the target device is used to output a generated video, the generated video includes multiple groups of generated adjacent images, the vector fields of pixel point motion in the multiple groups of generated adjacent images match the optical flow field corresponding to the image set, and the preset format includes at least one of tabular data, image data and text data.

[0124] Optionally, the processor 501 is further configured to implement the following steps when executing a program stored in the memory 503:

[0125] Performing feature recognition on at least two images in the image set to determine feature information of the video file;

[0126] The plurality of preset speed levels are acquired according to the characteristic information of the video file, wherein the characteristic information of the object described by each preset speed level in the plurality of preset speed levels matches the characteristic information of the video file.

[0127] Optionally, the processor 501 is further configured to implement the following steps when executing a program stored in the memory 503:

[0128] Reading continuous frames of the acquired video file at a preset time interval to obtain frame data;

[0129] The frame data is preprocessed to obtain the image set, wherein the preprocessing includes standardization of image size and image color.

[0130] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0131] The communication interface is used for communication between the above terminal and other devices.

[0132] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0133] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0134] In another embodiment provided by the present invention, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the motion speed information generating method described in any of the above embodiments.

[0135] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the motion speed information generating method described in any one of the above embodiments.

[0136] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0137] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0138] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for generating motion speed information, characterized in that: The method comprises: Based on the acquired video file, an image set is determined, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer; Calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images; According to the first optical flow value of the optical flow field, a target speed grade is determined among multiple preset speed grades, each of the preset speed grades corresponds to a preset second optical flow value, and the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the movement speed of the video file.

2. The method according to claim 1, characterized in that The step of calculating the movement speed of the N pixels according to the movement positions of the N pixels in the adjacent images and the time interval between the adjacent images to obtain the optical flow field corresponding to the image set includes: According to the moving positions between N pixels in the first image and N pixels in the second image, and the time interval between the first image and the second image, the moving speed of the N pixels along the target direction is calculated to obtain the moving speed of the N pixels in a plurality of groups of adjacent images along the target direction, wherein the first image and the second image are any groups of adjacent images in the image set; The movement speeds of N pixels in the plurality of groups of adjacent images along the target direction are combined to obtain an optical flow field corresponding to the image set.

3. The method according to claim 1, characterized in that The step of determining a target speed classification from a plurality of preset speed classifications according to the first optical flow value of the optical flow field comprises: When the direction of the first optical flow value of the optical flow field is the same as the direction of the preset second optical flow value, respectively calculating the optical flow value differences between the first optical flow value of the optical flow field and the second optical flow values ​​corresponding to a plurality of preset speed classifications, to obtain a plurality of optical flow value differences; The preset speed classification corresponding to the minimum optical flow value difference among the multiple optical flow value differences is determined as the target speed classification.

4. The method according to claim 1, characterized in that: After determining a target speed classification among a plurality of preset speed classifications according to the first optical flow value of the optical flow field, the method further includes: After associating the target speed classification with the optical flow field corresponding to the image set, the target device is used to output a generated video, the generated video includes multiple groups of generated adjacent images, the vector fields of pixel point motion in the multiple groups of generated adjacent images match the optical flow field corresponding to the image set, and the preset format includes at least one of tabular data, image data and text data.

5. The method according to claim 1, characterized in that According to the first optical flow value of the optical flow field, before determining a target speed classification among a plurality of preset speed classifications, the method further includes: Performing feature recognition on at least two images in the image set to determine feature information of the video file; The plurality of preset speed levels are acquired according to the characteristic information of the video file, wherein the characteristic information of the object described by each preset speed level in the plurality of preset speed levels matches the characteristic information of the video file.

6. The method according to claim 1, characterized in that Before obtaining the video file based on the method and determining the image set, the method further includes: Reading continuous frames of the acquired video file at a preset time interval to obtain frame data; The frame data is preprocessed to obtain the image set, wherein the preprocessing includes standardization of image size and image color.

7. A motion speed information generating device, characterized in that: The device comprises: A first acquisition module is used to determine an image set based on the acquired video file, wherein the image set includes a plurality of images arranged in time sequence, the time intervals between adjacent images in the plurality of images are the same, and each of the plurality of images includes N pixels, where N is a positive integer; A calculation module, used to calculate the movement speed of N pixels according to the movement positions of N pixels in adjacent images and the time interval between the adjacent images, so as to obtain an optical flow field corresponding to the image set, wherein the optical flow field is used to represent a vector field of the movement of pixels in multiple groups of adjacent images; A generating module is used to determine a target speed grade among a plurality of preset speed grades according to the first optical flow value of the optical flow field, each of the preset speed grades corresponds to a preset second optical flow value, the difference between the second optical flow value corresponding to the target speed grade and the first optical flow value is smaller than the difference between the second optical flow value corresponding to other preset speed grades and the first optical flow value, and the target speed grade is used to represent the motion speed of the video file.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store programs; A processor, configured to implement the method according to any one of claims 1 to 6 when executing a program stored in a memory.

9. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 6.

Images