Video data processing method, device, equipment and computer-readable storage medium

Abstract

The present application provides a video data processing method, apparatus, device, and computer-readable storage medium. The method comprises: acquiring video data and extracting multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person; detecting each of the person images through a preset skeletal key point detection network to obtain multiple skeletal key points in each of the person images; determining the walking information of the target person based on the multiple skeletal key points in each of the person images, wherein the walking information includes at least one of walking status, walking duration, walking distance, and walking speed; and determining the walking ability index of the target person based on the walking information. The present application relates to data analysis, and does not require human intervention during the entire video data processing process, greatly improving the accuracy and speed of human walking ability assessment.

Description

Technical Field

[0001] The present application relates to the technical field of data analysis, and in particular to a video data processing method, apparatus, device, and computer-readable storage medium. Background Art

[0002] In fields such as insurance and medical care, there are situations where a person's walking ability needs to be evaluated. For example, when purchasing insurance, the insured's walking ability needs to be evaluated. Insurance can only be purchased if the insured's walking ability meets the insurance conditions. For example, when undergoing health rehabilitation, by evaluating the rehabilitation person's walking ability, doctors and rehabilitation personnel can understand the rehabilitation progress and results.

[0003] However, existing assessments of a person's walking ability are primarily conducted by professional assessors following a complex process that requires the assessor's presence throughout the entire process. This results in high labor costs, slow assessments, and questionable accuracy. Therefore, accurately and quickly assessing a person's walking ability is a pressing issue. Summary of the Invention

[0004] The main purpose of this application is to provide a video data processing method, device, equipment and computer-readable storage medium, aiming to accurately and quickly evaluate a person's walking ability.

[0005] In a first aspect, the present application provides a method for processing video data, the method comprising the following steps:

[0006] Acquire video data, and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person;

[0007] Detecting each of the character images using a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images;

[0008] Determining walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed;

[0009] The walking ability index of the target person is determined according to the walking information.

[0010] In a second aspect, the present application further provides a video data processing device, the video data processing device comprising:

[0011] An extraction module is used to obtain video data and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking situation information of the target person;

[0012] A detection module is used to detect each of the character images through a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images;

[0013] a first determining module, configured to determine walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed;

[0014] The second determining module is used to determine the walking ability index of the target person according to the walking information.

[0015] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the video data processing method described above are implemented.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the video data processing method described above are implemented.

[0017] The present application provides a video data processing method, apparatus, device and computer-readable storage medium. The present application can accurately extract multiple person images from video data through a target detection network, and accurately obtain multiple skeletal key points in each person image through a skeletal key point detection network. Then, the walking information of the target person is determined based on the multiple skeletal key points in each person image, and the walking ability index of the target person can be accurately and quickly determined based on the walking information. The entire video data processing process does not require human intervention, which greatly improves the accuracy and speed of human walking ability assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technical personnel in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 A flowchart of a video data processing method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a rectangular frame in an embodiment of the present application;

[0021] Figure 3 for Figure 1 A schematic flow chart of sub-steps of the video data processing method in FIG.

[0022] Figure 4 A schematic diagram of a scenario for implementing the video data processing method provided in this embodiment;

[0023] Figure 5 A flowchart of another video data processing method provided in an embodiment of the present application;

[0024] Figure 6 A schematic block diagram of a video data processing device provided in an embodiment of the present application;

[0025] Figure 7 for Figure 5 A schematic block diagram of submodules of a video data processing device in FIG.

[0026] Figure 8 A schematic block diagram of another video data processing device provided in an embodiment of the present application;

[0027] Figure 9 This is a schematic block diagram of the structure of a computer device involved in one embodiment of the present application.

[0028] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0029] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary skilled persons in the art without making any creative efforts are within the scope of protection of this application.

[0030] The flowcharts shown in the accompanying drawings are illustrative only and do not necessarily include all content and operations / steps, nor do they necessarily need to be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual order of execution may vary depending on the actual situation. In addition, although the functional modules are divided in the device schematics, in some cases, the module division may be different from that shown in the device schematics.

[0031] Embodiments of the present application provide a video data processing method, apparatus, device, and computer-readable storage medium. The video data processing method can be applied to electronic devices or servers, such as mobile phones, tablet computers, laptop computers, desktop computers, personal digital assistants, and wearable devices. The server can be a single server or a server cluster consisting of multiple servers. The following explanation uses the video data processing method applied to a server as an example.

[0032] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0033] Please refer to Figure 1 , Figure 1 A flowchart of a video data processing method provided in an embodiment of the present application.

[0034] like Figure 1 As shown, the video data processing method includes steps S101 to S104.

[0035] Step S101: Acquire video data, and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person.

[0036] When video data of a target person is needed, the user can record the target person's walking process using an electronic device, thereby obtaining video data containing information about the target person's walking condition. The user can directly connect to the electronic device through a server, or transmit the video data recorded by the electronic device to the server, so that the server can evaluate the walking ability of the target person in the video data. The target person is the person whose walking ability is to be evaluated. It should be noted that the electronic device includes but is not limited to electronic devices with recording functions, such as video recorders, video cameras, and digital cameras.

[0037] In one embodiment, a user records the walking process of a target person through an electronic device, and stores the recorded video data in a database. When the server detects a walking ability assessment request triggered by the user, the server extracts the video data from the database based on the video identifier in the walking ability assessment request. The video identifier uniquely corresponds to the video data. Alternatively, the electronic device directly sends the recorded video data to the server, and the server receives it in real time or periodically. The video data may also be included in the walking ability assessment request. The server performs real-time evaluation and analysis on the received video data and outputs the results of the evaluation and analysis. It should be noted that the above-mentioned electronic devices include but are not limited to mobile phones, tablet computers, laptops, etc., and the database can be a local database or a cloud database.

[0038] When recording a target person's walking process using an electronic device, the electronic device displays a video recording page with a reminder message, which reminds the user that the target person in the video recording is within the video recording page. The user should pay attention to adjusting the shooting angle and shooting distance to ensure that the target person in the captured video data remains intact. When the user triggers a recording command based on the video recording page, the electronic device detects the recording command and records the target person's walking process based on the recording command, obtaining video data including information about the target person's walking situation.

[0039] After the server acquires the video data, a preset object detection network is used to extract multiple person images from the video data. The video data is composed of a plurality of frames of images, each of which includes walking information of a target person, the walking information recording the target person's walking process, and the person images include images of the person extracted from the plurality of frames. It should be noted that the preset object detection network can automatically detect the person in each frame of the video data, and the preset object detection network can extract at least one person image from each frame containing a person.

[0040] Among them, the above-mentioned preset target detection network is an improved target detection network, which is obtained by performing transfer learning on the traditional target detection network. The model sample data set for transfer learning of the improved target detection network is a data set of people labeled with a large amount of data, and optionally, it is obtained from the MSCOCO database of the data set of people labeled with a large amount of data.

[0041] Specifically, the process of transfer learning for the object detection network is as follows: during the model design phase, the convolutional layer parameters of the traditional object detection network are retained, and the model parameters corresponding to the model layers other than the convolutional layers are used as the model parameters to be trained; using the obtained model sample dataset, the model parameters corresponding to the model layers other than the convolutional layers are iteratively trained until the model converges. Once the model converges, the improved object detection network is obtained. Because the improved object detection network is trained based on a dataset with labeled people, the improved object detection network only includes the ability to detect people, and can accurately and quickly detect the location of people in images, greatly improving the efficiency of extracting human images.

[0042] In one embodiment, an object detection network includes a first object detection subnetwork and a second object detection subnetwork. A specific method for extracting multiple human images from video data is as follows: each frame of the video data is input into the first and second object detection subnetworks, respectively, to obtain a first rectangular frame and a second rectangular frame for each frame; a target rectangular frame for each frame is determined based on the first and second rectangular frames; and at least one human image is extracted from each frame based on the target rectangular frame. The first and second rectangular frames are used to select a human image in each frame of the video data, and the target rectangular frame is used to select a human image in each frame of the video data. If a human image exists in each frame, then at least one target rectangular frame exists in each frame.

[0043] It should be noted that the first and second object detection subnetworks are two improved object detection subnetworks, both of which were obtained through transfer learning from traditional object detection networks. The specific transfer learning process is described above and is not repeated here. The first and second object detection subnetworks enable more accurate extraction of human images from video data.

[0044] Furthermore, the specific method of determining the target rectangular frame of each frame image is: respectively obtaining the position information of the first rectangular frame and the position information of the second rectangular frame of each frame image; comparing the position information of the first rectangular frame of each frame image with the position information of the corresponding second rectangular frame to obtain the rectangular frame comparison result of each frame image; and determining the target rectangular frame of each frame image based on the rectangular frame comparison result of each frame image, the position information of the first rectangular frame and the position information of the second rectangular frame.

[0045] Among them, the position information of the first rectangular frame includes four first position coordinates of the four corner points of the first rectangular frame, and the position information of the second rectangular frame includes four second position coordinates of the four corner points of the second rectangular frame. The four corner points of the first rectangular frame in each frame image correspond one-to-one to the four corner points of the second rectangular frame, and the four first position coordinates correspond one-to-one to the four second position coordinates.

[0046] Specifically, the four first position coordinates of the four corner points of the first rectangular frame and the four second position coordinates of the four corner points of the second rectangular frame are obtained; each first position coordinate is compared with the corresponding second position coordinate in each frame of the image to obtain four comparison results for each frame of the image; the four corresponding target position coordinates are determined according to the four comparison results of each frame of the image, and the target rectangular frame in each frame of the image is determined by the four target position coordinates. The comparison result includes the comparison results of the horizontal coordinate and the vertical coordinate, and the comparison result uniquely corresponds to the target position coordinate. It should be noted that the method for determining a target position coordinate can be: determining the larger horizontal coordinate and the smaller vertical coordinate in a comparison result, and using the larger horizontal coordinate and the smaller vertical coordinate as the horizontal coordinate and the vertical coordinate of the corresponding target position coordinate, so that the target rectangular frame determined by the selected target position coordinate has a larger area, so that the target rectangular frame includes the first rectangular frame and the second rectangular frame.

[0047] For example, Figure 2 As shown, a rectangular coordinate system is established with the lower left corner of each frame image in the video data as the origin, and the four corner points of the first rectangular frame K1 are A1, A2, A3 and A4 respectively, and the four corner points of the second rectangular frame K2 are B1, B2, B3 and B4 respectively; the position coordinates of A1 are compared with the position coordinates of B1, the position coordinates of A2 are compared with the position coordinates of B2, the position coordinates of A3 are compared with the position coordinates of B3, and the position coordinates of A4 are compared with the position coordinates of B4, and four comparison results including horizontal coordinates and vertical coordinates are obtained; the horizontal coordinate of A1 and the vertical coordinate of B1 are selected as the first target position coordinates, that is, Figure 2 In C1, select the vertical coordinate of A4 and the horizontal coordinate of B4 as the second target position coordinate. Figure 2 In C2, select the position coordinates of A2 as the third target position coordinates, select the position coordinates of B3 as the fourth target position coordinates, and the target rectangular frame K3 can be determined through C1, C2, A2 and B3.

[0048] Step S102: Detect each of the character images using a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images.

[0049] After obtaining multiple person images through the improved object detection network, the server then uses a preset skeletal keypoint detection network to detect multiple skeletal keypoints in each person image. These skeletal keypoints are human skeleton keypoints used to represent the human skeletal structure. The skeletal keypoint detection network is implemented based on a neural network.

[0050] It should be noted that the implementation method of the skeleton key point detection network is: collecting skeleton key point data from a database of human data sets annotated with large amounts of data as a model sample data set, wherein the database can be selected as the MSCOCO database; designing a skeleton key point detection network based on a neural network, wherein the neural network can be selected as a high-resolution network (HRNet); iteratively training the skeleton key point detection network through the model sample data set until convergence, thereby obtaining the skeleton key point detection network.

[0051] Furthermore, the multiple skeletal key points are improved skeletal key points, that is, on the basis of the seventeen skeletal key points of nose, eyes, ears, shoulders, elbows, hands, hips, knees, ankles, etc. in the MSCOCO database, a skeletal key point of the neck is added, making the detected human skeletal structure more accurate.

[0052] In one embodiment, when multiple skeletal key points are obtained in each person image, each skeletal key point needs to be tracked to ensure that the skeletal key points of the target person in each subsequent person image are not lost. Taking tracking a skeletal key point in a person image as an example, the tracking method is specifically as follows: numbering each skeletal key point at different positions of the human body, wherein the numbering includes the number of each person image and the number of each skeletal key point in each person image; selecting a skeletal key point in a person image as a target skeletal key point, and obtaining each skeletal key point in the next person image frame of the person image; based on a preset sparse optical flow algorithm, calculating the key point similarity between the target skeletal key point and each skeletal key point in the next person image frame according to the target skeletal key point and each skeletal key point in the next person image frame; determining the maximum key point similarity between the target skeletal key point and each skeletal key point in the next person image frame, and using the skeletal key point in the next person image frame corresponding to the maximum key point similarity as the target skeletal key point to be tracked; continuing to obtain each skeletal key point in the next person image frame to track the target skeletal key point to be tracked until the last person image frame is tracked.

[0053] It should be noted that a skeletal key point selected in a person image can be selected as any skeletal key point in the first person image, which is convenient for tracking the entire walking process of the target person. The preset sparse optical flow algorithm can be set according to actual conditions, and this application does not make specific restrictions here. The skeletal key point in the next frame of the person image corresponding to the maximum key point similarity is selected as the target skeletal key point to be tracked. It can ensure that the skeletal key point in the next frame of the person image selected is correct. Using it as the target skeletal key point to be tracked can continue to track each skeletal key point in the next frame of the person image of the target skeletal key point, ensuring the continuity of tracking the skeletal key point at this position.

[0054] Among them, the calculation formula of key point similarity is:

[0055]

[0056] Among them, OKS p It represents the key point similarity between the skeleton key point numbered i in the character image and a skeleton key point in the next frame character image (hereinafter referred to as between two skeleton key points), d pi Represents the Euclidean distance of the skeleton key point numbered i in the character image, represents the area occupied by the target person in the image, σ i is the standard deviation between two skeleton key points, v pi is the attribute of the key point of the skeleton, δ(v pi =1) is the Kronecker function between two skeleton key points. The attributes of skeleton key points include invisible and visible.

[0057] Step S103: determining walking information of the target person based on the multiple skeleton key points in each of the person images, wherein the walking information includes at least one of walking status, walking duration, walking distance and walking speed.

[0058] After obtaining multiple skeleton key points in each person image, the server analyzes the multiple skeleton key points of the target person in each person image to determine the target person's walking information, wherein the walking information includes at least one of walking status, walking duration, walking distance, and walking speed.

[0059] In one embodiment, if Figure 3 As shown, step S103 includes: sub-step S1031 to sub-step S1032.

[0060] Sub-step S1031: determining multiple skeletal key points of the target person according to the multiple skeletal key points in each of the person images.

[0061] After obtaining multiple skeleton key points in each person image, the skeleton key points of the target person in each person image are screened out from the multiple skeleton key points in each person image, and the walking information of the target person is determined based on the multiple skeleton key points of the target person.

[0062] Sub-step S1032: determining walking information of the target person based on multiple skeletal key points of the target person.

[0063] After obtaining multiple skeletal key points of a target person, the target person's walking information can be determined based on these multiple skeletal key points. Specifically, any one of the target person's skeletal key points is selected as a target skeletal key point, and the position information of the target skeletal key point in each person image is obtained; based on the position information of the target skeletal key point in each person image, the target person's walking distance is determined; the time required for the walking distance to reach a preset distance threshold is calculated, and the time required for the walking distance to reach the preset distance threshold is used as the walking time of the target person. The position information includes the coordinate information of the target skeletal key point. The target skeletal key point can be selected as a skeletal key point whose position changes little during walking, such as a skeletal key point at the neck or shoulder.

[0064] It should be noted that the preset distance threshold can be set according to actual conditions and is not specifically limited in this application. It can be set to 45 meters. In some scenarios, the space size of the video data processing site does not meet the preset distance threshold. In this case, the target person's walking distance can be calculated based on the distance they travel back and forth. By using the time required for the walking distance to reach the preset distance threshold as the target person's walking time, and using the target person's walking time to evaluate the target person's walking ability, the evaluation result is more accurate and eliminates the influence of the target person's unstable speed during the walking process.

[0065] It is understandable that the specific method for determining the walking information of the target person can also be: selecting multiple skeletal key points of the target person as a skeletal key point set, and obtaining the position of each skeletal key point in the skeletal key point set in each person image; determining the walking distance of the target person corresponding to each skeletal key point based on the position of each skeletal key point in each person image; calculating the target walking time corresponding to each walking distance reaching a preset distance threshold, and obtaining the average of the target walking time as the walking time of the target person. By selecting multiple skeletal key points and obtaining the average walking time of the target person corresponding to each skeletal key point, the walking time of the determined target person can be made more accurate, indirectly improving the accuracy of evaluating the walking ability index of the target person.

[0066] In one embodiment, based on the multiple skeletal key points in each of the person images, the multiple skeletal key points of the target person and the multiple skeletal key points of persons other than the target person are determined. Based on the multiple skeletal key points of the target person and the multiple skeletal key points of persons other than the target person, the walking state of the target person is determined, wherein the walking state includes an assisted walking state and an independent walking state. By judging the walking state of the target person, it is determined whether the target person has assistance from other persons in the walking situation information. If the target person has assistance from other persons in the walking situation information, the walking ability index of the subsequent target person is correspondingly reduced, which can further improve the accuracy of video data processing.

[0067] Furthermore, the method for determining the walking state of the target person during the walking process is specifically as follows: determining whether the skeletal key points of the target person overlap with the skeletal key points of persons other than the target person in each person image; if the skeletal key points of the target person overlap with the skeletal key points of persons other than the target person, then determining that the walking state of the target person during the walking process is an auxiliary walking state; if the skeletal key points of the target person do not overlap with the skeletal key points of persons other than the target person, then determining that the walking state of the target person during the walking process is an independent walking state.

[0068] Step S104: Determine the target person's walking ability index based on the walking information.

[0069] After obtaining the walking information of the target person, the walking ability index of the target person can be determined based on the walking information of the target person. It should be noted that the walking ability index can be represented by a number or a level. For example, the walking ability index is represented by a number. Optionally, the numerical range of the walking ability index is set to 0-100. It can be understood that the larger the value of the walking ability index, the better the walking ability of the target person, and the smaller the value of the walking ability index, the worse the walking ability of the target person.

[0070] Specifically, based on the walking information of the target person, the walking ability index of the target person can be determined by including a mapping relationship table between the walking information and the walking ability index. The mapping relationship table between the walking information and the walking ability index can be set according to actual conditions and is not specifically limited in this application.

[0071] In one embodiment, based on the walking status of the target person in the walking information, the target person's assisted walking state duration and independent walking state duration are determined; the target person's total walking time is obtained, and the ratio of the assisted walking state duration and independent walking state duration to the total walking time are calculated; and the target person's walking ability index is determined based on the total walking time and the ratio of the assisted walking state duration to the total walking time. It should be noted that the total walking time is the time required for the target person's walking distance to reach a preset distance threshold, which can be 45. The assisted walking state duration is the duration of the total walking time occupied by the assisted walking state, and the independent walking state duration is the duration of the total walking time occupied by the independent walking state. Accordingly, the longer the total walking time, the lower the target person's walking ability index, and the greater the ratio of the assisted walking state duration to the total walking time, the lower the target person's walking ability index. The target person's walking ability index can be directly and quickly obtained based on the target person's walking status.

[0072] Specifically, the initial walking ability index corresponding to the total walking time and the weight coefficient corresponding to the proportion of the assisted walking state time to the total walking time are obtained, and the product of the initial walking ability index and the weight coefficient is calculated, and the product of the initial walking ability index and the weight coefficient is used as the walking ability index of the target person. It should be noted that the initial walking ability index corresponding to the total walking time and the weight coefficient corresponding to the proportion of the assisted walking state time to the total walking time can be set based on actual conditions, and this application does not make specific restrictions on this. For example, the assisted walking state time and the independent walking state time of the target person are 20s and 40s respectively, and the total walking time of the target person is 60s, then the proportion of the assisted walking state time and the independent walking state time to the total walking time are 33.3% and 66.7% respectively. When the target person's total walking time is between 58s and 63s, the target person's initial walking ability index is 80. When the proportion of the assisted walking state time to the total walking time is between 30% and 35%, the above weight coefficient is 0.82, that is, the target person's walking ability index is the product of the initial walking ability index 80 and the weight coefficient 0.82, so the target person's walking ability index is 65.6.

[0073] In one embodiment, the target person's walking ability index is determined based on the target person's walking time corresponding to a preset walking distance threshold as shown in the walking information. Specifically, a pre-stored mapping table between walking time and walking ability index is queried, and the walking ability index corresponding to the walking time is used as the target person's walking ability index. It is understood that the target person's walking ability index may also be determined based on the target person's walking speed corresponding to a preset walking distance threshold as shown in the walking information. Specifically, a pre-stored mapping table between walking speed and walking ability index is queried, and the walking ability index corresponding to the walking speed is used as the target person's walking ability index.

[0074] Please refer to Figure 4 , Figure 4 A schematic diagram of a scenario for implementing the video data processing method provided in this embodiment.

[0075] like Figure 4 As shown, the user can use the electronic device to record the target person's walking process, thereby obtaining video data including the target person's walking situation information. The electronic device can also directly obtain the video data, and the user then sends the video data to the server through the electronic device, and the server evaluates the walking ability of the target person in the video data.

[0076] The video data processing method provided in the above embodiment can accurately extract multiple character images from video data through the target detection network, and accurately obtain multiple skeleton key points in each character image through the skeleton key point detection network, and then determine the walking information of the target person based on the multiple skeleton key points in each character image, and accurately and quickly determine the walking ability index of the target person based on the walking information. The entire video data processing process does not require human intervention, which greatly improves the accuracy and speed of human walking ability assessment.

[0077] Please refer to Figure 5 , Figure 5 A flowchart of another video data processing method provided in an embodiment of the present application.

[0078] like Figure 5 As shown, the video data processing method includes steps S201 to S206.

[0079] Step S201: Obtain video data, and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person.

[0080] After the server acquires the video data, a preset object detection network is used to extract multiple person images from the video data. The video data is composed of a plurality of frames of images, each of which includes walking information of a target person, the walking information recording the target person's walking process, and the person images include images of the person extracted from the plurality of frames. It should be noted that the preset object detection network can automatically detect the person in each frame of the video data, and the preset object detection network can extract at least one person image from each frame containing a person.

[0081] In one embodiment, the target detection network includes a first target detection subnetwork, a second target detection subnetwork, and a border verification layer. The specific method for extracting multiple human images from video data is as follows: each frame of the video data is input into the first target detection subnetwork and the second target detection subnetwork, respectively, to obtain a first rectangular frame and a second rectangular frame for each frame; based on the border verification layer, the first rectangular frame is verified using the second rectangular frame, and it is determined whether the first rectangular frame passes the verification; if the first rectangular frame passes the verification, the target border is determined based on the positional relationship between the first rectangular frame and the second rectangular frame; and based on the target border, the human image is extracted from each frame. It should be noted that the above verification results include pass and fail, and the above border verification layer is used to verify the first rectangular frame.

[0082] Among them, the method of verifying the first rectangular frame through the second rectangular frame is specifically as follows: obtaining the position coordinates of the four corner points of the first rectangular frame, recorded as the first position coordinates of the four first corner points, and obtaining the second position coordinates of the four corner points of the second rectangular frame, recorded as the second position coordinates of the four second corner points, the first corner points and the second corner points correspond one to one, and the first position coordinates and the second position coordinates correspond one to one; calculating the coordinate difference between each first position coordinate and the corresponding second position coordinate to obtain four coordinate differences, and judging whether the four coordinate differences are less than or equal to the preset threshold value, if the four coordinate differences are less than or equal to the preset threshold value, it is determined that the first rectangular frame has passed the verification, and if at least one of the four coordinate differences is greater than the preset threshold value, it is determined that the first rectangular frame has not passed the verification. It should be noted that the above-mentioned preset threshold value can be set based on actual conditions, and this solution does not make specific restrictions on this.

[0083] Furthermore, according to the positional relationship between the first rectangular box and the second rectangular box, the target rectangular box is determined as follows: if the first rectangular box passes the verification, the positional relationship between the first rectangular box and the second rectangular box is determined, wherein the positional relationship includes an intersection relationship, an overlap relationship, an inclusion relationship and an irrelevant relationship; if the positional relationship between the first rectangular box and the second rectangular box is an intersection relationship, the border formed by the combination of the first rectangular box and the second rectangular box is used as the target border; if the positional relationship between the first rectangular box and the second rectangular box is an inclusion relationship, the first rectangular box or the second rectangular box containing one party is used as the target border; if the positional relationship between the first rectangular box and the second rectangular box is an overlap relationship, the first rectangular box or the second rectangular box is used as the target border; if there is no overlap between the positions of the first rectangular box and the second rectangular box, that is, the positional relationship between the first rectangular box and the second rectangular box is an irrelevant relationship, the areas of the first rectangular box and the second rectangular box are compared, and the first rectangular box or the second rectangular box with the larger area is used as the target border. It should be noted that the first rectangular frame or the second rectangular frame with a larger output area includes the image of the target person to be evaluated. By determining the positional relationship between the first rectangular frame and the second rectangular frame, the target frame is determined to avoid interference from other backgrounds, so that the extracted person image is more accurate.

[0084] In one embodiment, if the first rectangular box fails the verification, the corresponding person image is re-passed through the first target detection sub-network and the second target detection sub-network, and is verified again based on the border verification layer until the first rectangular box passes the verification, thereby ensuring that the person image can be extracted from each frame image in the video data.

[0085] Step S202: Detect each of the character images using a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images.

[0086] After obtaining multiple person images through the improved object detection network, the server then uses a preset skeletal keypoint detection network to detect multiple skeletal keypoints in each person image. These skeletal keypoints are human skeleton keypoints used to represent the human skeletal structure. The skeletal keypoint detection network is implemented based on a neural network.

[0087] Step S203: Determine walking information of the target person according to the multiple skeleton key points in each of the person images.

[0088] After obtaining the multiple skeleton key points in each person image, the skeleton key points of the target person in each person image are screened out from the multiple skeleton key points in each person image, and walking information of the target person is determined based on the multiple skeleton key points of the target person. The walking information of the target person includes at least one of the walking state, walking duration, walking speed, and walking distance of the target person.

[0089] In one embodiment, the walking information of the target person includes the standing state of the target person, and the standing state includes the upright state and the non-upright state of the target person. The trunk width-to-height ratio of the target person is calculated through the skeleton key points of the target person. When the trunk width-to-height ratio is less than the preset trunk width-to-height ratio, the target person is considered to be in the upright state. When the trunk width-to-height ratio is greater than or equal to the preset trunk width-to-height ratio, the target person is considered to be in the non-upright state. It should be noted that the preset trunk width-to-height ratio can be set according to actual conditions. This application does not make specific restrictions here, and it can be selected as 0.4. The evaluation process also records the skeleton key point data of each frame image. The movement changes of the skeleton key points of the entire evaluation process can form a playback video. Information such as whether the standing state of the target person is an upright state or a non-upright state can be displayed in the playback video after the evaluation. By determining the standing state of the target person, the walking situation of the target person can be further understood.

[0090] In one embodiment, the walking information of the target person includes the target person's body posture data. Based on the body posture data, the assessee's body curvature C, stride length W, and walking speed V are determined. The assessee's fall index Fe is determined based on the assessee's body curvature C, stride length W, and walking speed V. The body curvature C, which is the angle between the straight line formed by the neck and thigh joints and the straight line formed by the leg joints, is a dynamic value during the assessment process. The maximum detected angle is used as the body curvature C. The stride length W represents the maximum distance between the assessee's left and right feet while walking.

[0091] The calculation formula of the fall index Fe is:

[0092]

[0093] in, is the correction coefficient, generally set at around 100. The larger the body curvature V and stride length W, the smaller the fall index Fe. The larger the body curvature C, the larger the fall index Fe. This formula can map various data affecting falls to a range of 0 to 1 after calculation. It should be noted that the fall index indicates the target person's susceptibility to falls. When subsequently determining the target person's walking ability index, the fall index can be used as an influencing factor of the target person's walking ability index. It is understandable that the higher the fall index, the more likely the target person is to fall, and the lower the walking ability index, the less likely the target person is to fall, and the higher the walking ability index.

[0094] Step S204: Determine a first walking ability index of the target person according to the walking status in the walking information.

[0095] A first walking ability index of the target person corresponding to the walking state is determined based on the walking state in the walking information of the target person. Specifically, the walking state of the target person in each person image is determined to be an assisted walking state or an independent walking state, and the first walking ability index of the corresponding target person is determined based on the assisted walking state and / or independent walking state in the walking information of the target person, wherein the first walking ability index is a preliminarily determined walking ability index.

[0096] In one embodiment, whether the target person is in an assisted walking state in the walking information is determined based on multiple skeletal key points of the target person and multiple skeletal key points of persons other than the target person; if the target person is in an assisted walking state in the walking information, the assisted walking duration of the target person in the walking information is determined based on the skeletal key points of the target person in each person image and the skeletal key points of persons other than the target person in each person image; and the first walking ability index of the target person is determined based on the assisted walking duration in the walking information.

[0097] Specifically, the total walking time of the video data of the walking information is obtained, and the percentage of the assisted walking time to the total walking time is calculated. The difference between 1 and the percentage is multiplied by a preset walking index to obtain the first walking ability index of the target person. The preset walking index can be set according to actual conditions and is not specifically limited in this application. It can be selected as 100.

[0098] For example, the assisted walking time of the target person in the walking information is 20 seconds, the total walking time of the video data during the walking process is 30 seconds, the percentage of the assisted walking time in the total walking time is 60%, the absolute value of the difference between 100% and the percentage is 40%, and the preset walking index is 100. Therefore, the first walking ability index is the absolute value of the difference multiplied by the preset walking index, which is equal to 40.

[0099] In one embodiment, if the target person's walking information does not include an assisted walking state, that is, if the walking condition information indicates an independent walking state, that is, if there is no assisted walking ability index, the target person's first walking ability index is a preset walking index. This walking index can be set according to actual conditions and is not specifically limited in this application. It can be set to 100.

[0100] In one embodiment, if the target person is in a completely assisted walking state when walking, it is understandable that the first walking ability index of the target person is zero.

[0101] Step S205: Determine a second walking ability index of the target person according to the walking speed in the walking information.

[0102] After determining the first walking ability index of the target person, the average walking speed of the target person is determined based on the skeletal key points of the target person in each character image, and the second walking ability index of the target person is determined based on the average walking speed of the target person. That is, a pre-stored mapping relationship table between walking speed and walking ability index is obtained, and the mapping relationship table is queried, and the walking ability index corresponding to the average walking speed is used as the second walking ability index of the target person. It should be noted that the mapping relationship table between the above-mentioned walking speed and walking ability index can be set based on actual conditions, and this application does not make specific restrictions on this. Obtaining the average walking speed of the target person can reduce the instability caused by walking speeds of different sizes, so that the determined second walking ability index of the target person is more accurate.

[0103] For example, the average walking speed of the target person is 0.5 m / s. According to the preset mapping relationship table, the second walking ability index corresponding to the average walking speed is 75. Therefore, the second walking ability index of the target person in this walking ability assessment is 75.

[0104] The method for determining the target person's average walking speed is as follows: a random skeleton key point is selected as the target skeleton key point, and two different frames of the person's image are randomly selected. The position of the target skeleton key point is obtained from the two selected frames of the person's image, and two position coordinates are obtained through weighted averaging. The time difference between the two selected person images is calculated, and the distance walked by the target person within the time difference is determined based on the two obtained position coordinates. The average walking speed of the target person is calculated based on the walking distance and the time difference. By determining the average walking speed of the target person, the results of the walking ability assessment are more accurate.

[0105] Step S206: Determine the walking ability index of the target person according to the first walking ability index and the second walking ability index.

[0106] According to the first walking ability index and the second walking ability index, the walking ability index of the target person can be comprehensively determined. Specifically, based on the preset first weight coefficient and the second weight coefficient, the weighted average of the first walking ability index and the second walking ability index is calculated to obtain the walking ability index of the target person. The specific method for obtaining the above-mentioned weighted average is: the product of the first weight coefficient and the first walking ability index is added to the second weight coefficient and the second walking ability index to obtain the walking ability index of the target person. It should be noted that the preset first weight coefficient and the second weight coefficient can be set according to the specific situation, and this application does not make specific restrictions here. Optionally, the first weight coefficient and the second weight coefficient are 0.4 and 0.6 respectively.

[0107] For example, the first weight coefficient and the second weight coefficient are 0.4 and 0.6 respectively, the first walking ability index is 40, and the second walking ability index is 75. By taking a weighted average of the first walking ability index and the second walking ability index, that is, adding the product of the first weight coefficient 0.4 and the first walking ability index 40 plus the second weight coefficient 0.6 and the second walking ability index 75, the target person's walking ability index can be obtained as 61.

[0108] The video data processing method provided in the above embodiment accurately obtains multiple skeletal key points in each character image through a target detection network and a skeletal key point detection network, determines the walking information of the target person based on the multiple skeletal key points in each character image, and determines the first walking ability index based on the walking status in the walking information, which can improve the accuracy of the subsequent walking ability index. Based on the preset mapping relationship table and the walking speed in the walking information, the corresponding second walking ability index can be determined. According to the first walking ability index and the second walking ability index, the walking ability index of the target person can be accurately determined, which greatly improves the accuracy of the person's walking ability assessment.

[0109] Please refer to Figure 6 , Figure 6 A schematic block diagram of a video data processing device provided in an embodiment of the present application.

[0110] like Figure 6 As shown, the video data processing device 300 includes: an extraction module 301 , a detection module 302 , a first determination module 303 and a second determination module 304 .

[0111] An extraction module 301 is configured to acquire video data and extract a plurality of person images from the video data using a preset target detection network, wherein the video data includes walking condition information of the target person;

[0112] The detection module 302 is used to detect each of the character images through a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images;

[0113] A first determining module 303 is configured to determine walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed;

[0114] The second determining module 304 is configured to determine the target person's walking ability index based on the walking information.

[0115] In one embodiment, the extraction module 301 is further configured to:

[0116] Inputting each frame of the video data into the first object detection subnetwork and the second object detection subnetwork respectively to obtain a first rectangular frame and a second rectangular frame of each frame of the video data;

[0117] Determine a target rectangular frame for each frame of image according to the first rectangular frame and the second rectangular frame of each frame of image;

[0118] At least one person image is extracted from each frame of image according to the target rectangular frame of each frame of image.

[0119] In one embodiment, the extraction module 301 is further configured to:

[0120] respectively obtaining the position information of the first rectangular frame and the position information of the second rectangular frame of each frame of image;

[0121] Comparing the position information of the first rectangular frame of each frame image with the position information of the corresponding second rectangular frame to obtain a rectangular frame comparison result of each frame image;

[0122] A target rectangular frame for each frame of image is determined according to the rectangular frame comparison result of each frame of image, the position information of the first rectangular frame, and the position information of the second rectangular frame.

[0123] In one embodiment, the first determining module 303 is further configured to:

[0124] Determining a plurality of skeletal key points of the target person and a plurality of skeletal key points of persons other than the target person according to a plurality of skeletal key points in each of the person images;

[0125] The walking state of the target person is determined according to the multiple skeleton key points of the target person and the multiple skeleton key points of people other than the target person.

[0126] In one embodiment, Figure 7As shown, the first determining module 303 includes:

[0127] A first determining submodule 3031 is configured to determine a plurality of skeletal key points of the target person according to the plurality of skeletal key points in each of the person images;

[0128] The second determining submodule 3032 is configured to determine walking information of the target person based on a plurality of skeletal key points of the target person.

[0129] In one embodiment, the second determining submodule 3032 is further configured to:

[0130] Selecting any one skeleton key point of the target person as a target skeleton key point, and obtaining position information of the target skeleton key point in each of the person images;

[0131] Determining the walking distance of the target person according to the position information of the target skeleton key point in each of the person images;

[0132] The time required for the walking distance to reach a preset distance threshold is calculated, and the time required for the walking distance to reach the preset distance threshold is used as the walking time of the target person.

[0133] Please refer to Figure 8 , Figure 8 A schematic block diagram of another video data processing device provided in an embodiment of the present application.

[0134] like Figure 8 As shown, the video data processing device 400 includes:

[0135] The extraction module 401 is used to obtain video data and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking situation information of the target person.

[0136] The detection module 402 is used to detect each of the character images through a preset skeleton key point detection network to obtain multiple skeleton key points in each of the character images.

[0137] The first determining module 403 is configured to determine the walking information of the target person according to a plurality of skeleton key points in each of the person images.

[0138] The second determining module 404 is configured to determine a first walking ability index of the target person according to the walking state in the walking information.

[0139] The third determining module 405 is configured to determine a second walking ability index of the target person according to the walking speed in the walking information.

[0140] The fourth determining module 406 is configured to determine the target person's walking ability index based on the first walking ability index and the second walking ability index.

[0141] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned video data processing method embodiment, and will not be repeated here.

[0142] The apparatus provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 9 Runs on the computer equipment shown.

[0143] See also Figure 9 , Figure 9 This is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server or an electronic device.

[0144] like Figure 9 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0145] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any video data processing method.

[0146] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0147] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any video data processing method.

[0148] The network interface is used for network communication, such as sending assigned tasks, etc. It can be understood by those skilled in the art that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0149] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0150] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0151] Acquire video data, and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person;

[0152] Detecting each of the character images using a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images;

[0153] Determining walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed;

[0154] The walking ability index of the target person is determined according to the walking information.

[0155] In one embodiment, when determining the walking information of the target person based on the multiple skeletal key points in each of the person images, the processor is configured to implement:

[0156] Determining a plurality of skeletal key points of the target person according to a plurality of skeletal key points in each of the person images;

[0157] Walking information of the target person is determined according to a plurality of skeleton key points of the target person.

[0158] In one embodiment, when determining the walking information of the target person based on the multiple skeletal key points in each of the person images, the processor is configured to implement:

[0159] Determining a plurality of skeletal key points of the target person and a plurality of skeletal key points of persons other than the target person according to a plurality of skeletal key points in each of the person images;

[0160] The walking state of the target person is determined according to the multiple skeleton key points of the target person and the multiple skeleton key points of the person other than the target person.

[0161] In one embodiment, when determining the walking information of the target person based on the multiple skeletal key points of the target person, the processor is configured to implement:

[0162] Selecting any one skeleton key point of the target person as a target skeleton key point, and obtaining position information of the target skeleton key point in each of the person images;

[0163] Determining the walking distance of the target person according to the position information of the target skeleton key point in each of the person images;

[0164] The time required for the walking distance to reach a preset distance threshold is calculated, and the time required for the walking distance to reach the preset distance threshold is used as the walking time of the target person.

[0165] In one embodiment, when determining the walking ability index of the target person based on the walking information and a preset mapping relationship table, the processor is configured to implement:

[0166] determining a first walking ability index of the target person according to the walking state in the walking information;

[0167] determining a second walking ability index of the target person according to the walking speed in the walking information;

[0168] The walking ability index of the target person is determined according to the first walking ability index and the second walking ability index.

[0169] In one embodiment, the processor, when implementing the target detection network including a first target detection subnetwork and a second target detection subnetwork; and extracting multiple person images from the video data through the preset target detection network, is configured to implement:

[0170] Inputting each frame of the video data into the first object detection subnetwork and the second object detection subnetwork respectively to obtain a first rectangular frame and a second rectangular frame of each frame of the video data;

[0171] A target rectangular frame of each frame of image is determined according to the first rectangular frame and the second rectangular frame of each frame of image; and at least one person image is extracted from each frame of image according to the target rectangular frame of each frame of image.

[0172] In one embodiment, when determining the target rectangular frame of each frame of image based on the first rectangular frame and the second rectangular frame of each frame of image, the processor is configured to implement:

[0173] respectively obtaining the position information of the first rectangular frame and the position information of the second rectangular frame of each frame of image;

[0174] Comparing the position information of the first rectangular frame of each frame image with the position information of the corresponding second rectangular frame to obtain a rectangular frame comparison result of each frame image;

[0175] A target rectangular frame for each frame of image is determined according to the rectangular frame comparison result of each frame of image, the position information of the first rectangular frame, and the position information of the second rectangular frame.

[0176] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned text processing method embodiment, and will not be repeated here.

[0177] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the video data processing method of the present application.

[0178] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., equipped on the computer device.

[0179] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0180] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0181] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims.

Claims

1. A video data processing method, characterized in that: include: Acquire video data, and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking condition information of the target person; Detecting each of the character images using a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images; Determining walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed; determining a walking ability index of the target person according to the walking information; The step of determining the target person's walking ability index based on the walking information includes: Determining a duration of the target person's assisted walking state based on the walking state; Obtaining the total walking time of the target person, and calculating the ratio of the assisted walking state time to the total walking time; Obtaining an initial walking ability index corresponding to the total walking time and a weight coefficient corresponding to the ratio of the assisted walking state time to the total walking time; The product of the initial walking ability index and the weight coefficient is calculated, and the product of the initial walking ability index and the weight coefficient is used as the walking ability index of the target person.

2. The video data processing method according to claim 1, wherein: The step of determining walking information of the target person according to the plurality of skeleton key points in each of the person images includes: Determining a plurality of skeletal key points of the target person according to a plurality of skeletal key points in each of the person images; Walking information of the target person is determined according to a plurality of skeleton key points of the target person.

3. The video data processing method according to claim 1, wherein: The step of determining walking information of the target person according to the plurality of skeleton key points in each of the person images includes: Determining a plurality of skeletal key points of the target person and a plurality of skeletal key points of persons other than the target person according to a plurality of skeletal key points in each of the person images; The walking state of the target person is determined according to the multiple skeleton key points of the target person and the multiple skeleton key points of people other than the target person.

4. The video data processing method according to claim 2, wherein: The step of determining walking information of the target person based on a plurality of skeletal key points of the target person includes: Selecting any one skeleton key point of the target person as a target skeleton key point, and obtaining position information of the target skeleton key point in each of the person images; Determining the walking distance of the target person according to the position information of the target skeleton key point in each of the person images; The time required for the walking distance to reach a preset distance threshold is calculated, and the time required for the walking distance to reach the preset distance threshold is used as the walking time of the target person.

5. The video data processing method according to any one of claims 1 to 4, characterized in that: The object detection network includes a first object detection subnetwork and a second object detection subnetwork; The step of extracting a plurality of person images from the video data by using a preset target detection network includes: Inputting each frame of the video data into the first object detection subnetwork and the second object detection subnetwork respectively to obtain a first rectangular frame and a second rectangular frame of each frame of the video data; Determine a target rectangular frame for each frame of image according to the first rectangular frame and the second rectangular frame of each frame of image; At least one person image is extracted from each frame of image according to the target rectangular frame of each frame of image.

6. The video data processing method according to claim 5, wherein: The step of determining a target rectangular frame for each frame of image according to the first rectangular frame and the second rectangular frame of each frame of image comprises: respectively obtaining the position information of the first rectangular frame and the position information of the second rectangular frame of each frame of image; Comparing the position information of the first rectangular frame of each frame image with the position information of the corresponding second rectangular frame to obtain a rectangular frame comparison result of each frame image; A target rectangular frame for each frame of image is determined according to the rectangular frame comparison result of each frame of image, the position information of the first rectangular frame, and the position information of the second rectangular frame.

7. A video data processing device, characterized in that: The video data processing device comprises: An extraction module is used to obtain video data and extract multiple person images from the video data through a preset target detection network, wherein the video data includes walking situation information of the target person; A detection module is used to detect each of the character images through a preset skeleton key point detection network to obtain a plurality of skeleton key points in each of the character images; a first determining module, configured to determine walking information of the target person based on a plurality of skeletal key points in each of the person images, wherein the walking information includes at least one of a walking state, a walking duration, a walking distance, and a walking speed; A second determining module is used to determine the walking ability index of the target person according to the walking information; Among them, the second determination module is also used to determine the assisted walking state duration of the target person based on the walking state; obtain the total walking time of the target person, and calculate the proportion of the assisted walking state duration to the total walking time; obtain the initial walking ability index corresponding to the total walking time and the weight coefficient corresponding to the proportion of the assisted walking state duration to the total walking time; calculate the product of the initial walking ability index and the weight coefficient, and use the product of the initial walking ability index and the weight coefficient as the walking ability index of the target person.

8. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the video data processing method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the video data processing method according to any one of claims 1 to 6 are implemented.

Images