Video processing method, device, electronic device and storage medium

Abstract

The present invention discloses a video processing method, device, electronic device and storage medium, belonging to the field of video processing technology. In view of the problem of video recognition, the present invention provides a video processing method, including responding to a received video review request, if the triggering condition of the video review request is reaching a first preset duration, adjusting the time range of video recognition according to the first preset duration, and dividing the subsequent video according to the duration of the adjusted video recognition time range as the time range of the next video recognition until the event is complete; and / or if the triggering condition of the video review request is receiving a recording end signal, confirming the duration of the complete event of the target tag, if the duration is not greater than the second preset duration, then the time range of video recognition is not adjusted, if the duration is greater than the second preset duration, then according to the duration of the complete event of the target tag, adjust the time range of video recognition.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video processing method, device, electronic equipment and storage medium. Background Art

[0002] Video recognition is to identify the information that meets the requirements in a specific video and mark it with a label in preparation for further judgment and presentation of the video. Video recognition is best performed on videos that contain a complete event, otherwise it will cause incoherent recognition, which will affect subsequent judgments. However, it is also not possible to recognize videos that are too long, otherwise there will be too much invalid data, which will also affect subsequent judgments. At the same time, events in some cases have pre-swings, so the video of the complete event should also include pre-swings. For example, in the field of pet videos, based on the user's operation of certain execution instructions, the pet appears in the recording range and leaves the recording range. The video should include the part from below the instruction to before the pet appears, so as to reflect the interaction between the user and the pet, increase the fun of the video, and enhance the user experience.

[0003] A complete event refers to the start and end of an event with a tag. For example, if the tag is movement, the complete event of movement is that when the device detects movement, it immediately triggers video recording, and stops recording n seconds after there is no movement, which is considered the complete end of the event; if the tag is pet, the complete event of pet is that when the device detects the presence of a pet, it immediately triggers video recording, and stops recording n seconds after the absence of the pet, which is considered the complete end of the event; if the tag is eating, the complete event of eating is that when the device detects a change in the data of the weighing bowl sensor, it immediately triggers video recording, and stops recording n seconds after there is no data change, which is considered the complete end of the event.

[0004] Pre-swing refers to instructions that are strongly related to the occurrence of an event. Generally speaking, instructions within a specific time range before the event is considered to be the start of the pre-swing of the event.

[0005] When there is a requirement for timeliness of recognition, if the duration of the complete event is long, recognition after the event is completed will significantly reduce timeliness. Therefore, it is necessary to first determine the appropriate time range for video recognition.

[0006] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0007] In order to solve the related problems, the present invention provides a video processing method, device, electronic device and storage medium. The technical solution is:

[0008] A video processing method, comprising:

[0009] In response to a received video review request, if the trigger condition of the video review request is reaching a first preset duration, the time range of video recognition is adjusted according to the first preset duration, and subsequent videos are divided according to the duration of the adjusted video recognition time range as the time range for the next video recognition until the event is complete; and / or if the trigger condition of the video review request is receiving a recording end signal, confirming the duration of the complete event of the target tag, if the duration is not greater than a second preset duration, the time range of video recognition is not adjusted; if the duration is greater than the second preset duration, the time range of video recognition is adjusted according to the duration of the complete event of the target tag.

[0010] In a preferred embodiment of the present invention, it also includes:

[0011] Receiving a video recognition request sent by the first platform, the video recognition request including a time range, a recognition frequency, and a target tag for video recognition, the recognition frequency being X frames per second, wherein the first platform sends the video recognition request in response to the received recording start signal;

[0012] In response to the video recognition request, retrieve and recognize the corresponding video to generate recognition data, and send the recognition data to the first platform, wherein the recognition data includes a plurality of tags;

[0013] A video review request is received from the first platform, wherein the first platform confirms whether the target tag is included in the multiple tags received, and if so, sends a video review request after the earlier of reaching a first preset time or receiving a recording end signal.

[0014] In a preferred embodiment of the present invention, if the triggering condition of the video review request is reaching a first preset duration, an abnormal instruction is reported.

[0015] In a preferred embodiment of the present invention, in response to receiving an instruction to adjust the start time of video recognition, the start time of the corresponding video recognition time range is moved forward to the adjusted start time; wherein, the instruction to adjust the start time of the video recognition refers to receiving an execution instruction within a third preset time period before the start time of an event of a target tag, and the video recognition start time is confirmed based on the time when the execution instruction is received.

[0016] In a preferred embodiment of the present invention, if the execution instruction is received within a third preset time before the event start time of a target tag, the video recognition start time is confirmed according to the time when the execution instruction is received, including:

[0017] The third preset time length is divided into n equal parts, and whether an execution instruction is received in a corresponding time period is detected one by one. If an execution instruction is received, the start time of the time period in which the execution instruction is received is used as the video recognition start time.

[0018] In a preferred embodiment of the present invention, the recognition frequency is adjusted in response to a received video review request and / or in response to a received instruction to adjust the video recognition start time.

[0019] In a preferred embodiment of the present invention, the corresponding video is retrieved and recognized according to the adjusted time range and recognition frequency of the video recognition to generate adjusted recognition data, and the adjusted recognition data is sent to the first platform.

[0020] Another technical solution is:

[0021] A video processing device, comprising:

[0022] A receiving module, used for receiving a video review request;

[0023] An adjustment module is used to adjust the time range of video recognition according to the first preset duration when the trigger condition of a video review request is reaching the first preset duration, and divide the subsequent videos according to the duration of the adjusted video recognition time range as the time range for the next video recognition until the event is complete; when the trigger condition of a video review request is receiving a recording end signal, confirm the duration of the complete event of the target tag, and when the duration is not greater than a second preset duration, do not adjust the time range of video recognition, and when the duration is greater than the second preset duration, adjust the time range of video recognition according to the duration of the complete event of the target tag.

[0024] Another technical solution is:

[0025] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0026] Another technical solution is:

[0027] A computer-readable storage medium stores a computer program for executing any of the above methods.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention achieves a balance between the completeness and timeliness of the recognition event by adjusting the time range of video recognition in response to a received video review request; adjusts the video recognition start time in response to a received instruction to adjust the video recognition start time to add the pre-swing to the recognition; and further adjusts the recognition frequency to achieve the adjustment of the recognition accuracy. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0032] Embodiment 1:

[0033] A video processing method, comprising:

[0034] A video recognition request sent by the first platform is received, where the video recognition request includes a time range, a recognition frequency, and a target tag for video recognition, where the recognition frequency is X frames per second, wherein the first platform sends the video recognition request in response to the received recording start signal.

[0035] In response to a video recognition request, the corresponding video is retrieved and recognized to generate recognition data, and the recognition data is sent to the first platform, wherein the recognition data includes a plurality of tags.

[0036] For example, the target label is has pets.

[0037] For video content that may contain pets, the video source file is identified at a frequency of X frames / second, for example, X is 1. After identification, each frame with content is labeled frame by frame and sent back to the first platform. The labels are as follows:

[0038] a) Device ID

[0039] b) Account ID

[0040] c) Document ID

[0041] d) Frame timestamp

[0042] e) Whether there is a pet (1 for yes, 0 for no)

[0043] f) Number of pets (1 means one pet, 2 means two pets, the number will be displayed based on the actual number of pets)

[0044] g) Pet type (1 for cat, 0 for dog)

[0045] h) Pet position (X axis, Y axis, hereinafter referred to as X, Y)

[0046] i) Pet size (width and height, hereinafter referred to as W and H)

[0047] j) Confidence

[0048] k) Screen scoring, which refers to the scoring of the screen structure, position and size

[0049] l) Whether there is human-pet interaction, which means that the video has both "motion detection" or "pet detection" and "backend user interaction behavior" (1 for yes, 0 for no)

[0050] For example, WSQ20221230F001, 29394, a(1)8478, 20221230142345, 1, 1, 1, X216Y39, W50H93, 70, 80, 1 means the following:

[0051] The device ID is WSQ20221230F001, the account ID is 29394, and the file ID is 8478. The mobile detection detected a cat with a width of 50 and a height of 93 at the coordinates (216, 39) at 14:23:45 on December 30, 2022. The confidence level is 70, the picture score is 80, and there is human-pet interaction.

[0052] A video review request sent by the first platform is received, wherein the first platform confirms whether the target tag is included in the multiple tags received, and if so, sends the video review request after the earlier of reaching the first preset time length or receiving the recording end signal.

[0053] For example, if it is confirmed that a target tag with a pet is received, a video review request is sent after the subsequent conditions are met.

[0054] In response to the received video review request, if the trigger condition of the video review request is reaching a first preset duration, the time range of video recognition is adjusted according to the first preset duration, and subsequent videos are divided according to the duration of the adjusted video recognition time range as the time range for the next video recognition until the event is complete.

[0055] If the trigger condition of the video review request is receiving a recording end signal and confirming the duration of the complete event of the target tag, if the duration is not greater than the second preset duration, the time range of video recognition will not be adjusted; if the duration is greater than the second preset duration, the time range of video recognition will be adjusted according to the duration of the complete event of the target tag.

[0056] For example, the first preset duration is 60 seconds. If the pet has not gone away until 60 seconds, that is, has not left the recording range, the event has not ended. The 60-second video will be reviewed first, and abnormal instructions can be reported, such as immediately notifying the user to watch, and reviewing subsequent videos every 60 seconds until the event is over.

[0057] Receiving the recording end signal means that the event is completely finished. If the duration of the complete event exceeds the second preset duration, such as 10 seconds, the recognition time range is adjusted to the actual start and end time of the complete event; if the duration of the complete event does not exceed 10 seconds, there is no need to adjust the recognition time range, thereby avoiding too little recognition data and facilitating subsequent adjustment of the recognition frequency. More preferably, the second preset duration is the same as the duration of the video recognition time range in the video recognition request, thereby reducing the adjustment of the recognition time range.

[0058] The deadline of the event, that is, the nth second at which recording stops, is preferably 30 seconds when the tag is a pet, that is, recording stops 30 seconds after the pet is gone, and the event is considered to be completely ended, thereby avoiding misjudgment caused by the pet's back and forth movement within the recording range. At the same time, the video integrity is high and the data volume is appropriate.

[0059] In response to receiving an instruction to adjust the start time of video recognition, the start time of the corresponding video recognition time range is moved forward to the adjusted start time; wherein, the instruction to adjust the start time of video recognition refers to receiving an execution instruction within a third preset time period before the start time of an event of a target tag, and the video recognition start time is confirmed according to the time when the execution instruction is received.

[0060] That is, it is determined whether there is a pre-shake before the event. In this embodiment, the pre-shake refers to the action of the user issuing a command to the device before the pet appears, so as to ensure the integrity of the event.

[0061] Specifically, the third preset time length can be divided into n equal parts, and whether an execution instruction is received in the corresponding time period can be detected one by one, thereby saving time. If an execution instruction is received, the start time of the time period in which the execution instruction is received is used as the video recognition start time.

[0062] For example, the third preset time length is 30 seconds, which is divided into three equal parts, that is, the start detection time is 10 seconds before the actual start time of the event, and the end time is still the actual end time of the event.

[0063] i. If the device has an execution instruction 10 seconds before the event start time, the video recognition start time will be adjusted forward 10 seconds;

[0064] ii. If the device does not have any execution instructions 10 seconds before the event start time, the start detection time is adjusted to 20 seconds before the actual start time of the event;

[0065] iii. If the device has an execution instruction 20 seconds before the event start time, the video recognition start time will be adjusted forward 20 seconds;

[0066] iv. If the device does not have any execution instructions 20 seconds before the event start time, the start detection time is adjusted to 30 seconds before the actual start time of the event;

[0067] v. If the device has an execution instruction 30 seconds before the event start time, the video recognition start time will be adjusted forward 30 seconds;

[0068] vi. If the device does not execute any instructions within 30 seconds before the start time of the event, the detection will be terminated.

[0069] In response to a received video review request and / or in response to a received instruction to adjust the video recognition start time, the recognition frequency is adjusted, for example, the recognition frequency may be adjusted to Y frames / second, where Y>X, thereby improving recognition accuracy.

[0070] According to the adjusted time range and recognition frequency of the video recognition, the corresponding video is retrieved and recognized to generate adjusted recognition data, and the adjusted recognition data is sent to the first platform.

[0071] Embodiment 2:

[0072] Based on Embodiment 1, this embodiment provides a video processing device, including:

[0073] The receiving module is used to receive a video review request.

[0074] An adjustment module is used to adjust the time range of video recognition according to the first preset duration when the trigger condition of a video review request is reaching a first preset duration, and divide the subsequent videos according to the duration of the adjusted video recognition time range as the time range for the next video recognition until the event is complete; when the trigger condition of a video review request is receiving a recording end signal, confirm the duration of the complete event of the target tag, and do not adjust the time range of video recognition when the duration is not greater than a second preset duration, and adjust the time range of video recognition according to the duration of the complete event of the target tag when the duration is greater than the second preset duration.

[0075] Embodiment 3:

[0076] Based on Embodiment 1, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. Embodiment 1 is implemented when the processor executes the program.

[0077] Embodiment 4:

[0078] A computer-readable storage medium stores a computer program for executing embodiment 1.

[0079] In summary, the present invention achieves a balance between the completeness and timeliness of the recognition event by adjusting the time range of video recognition in response to a received video review request; adjusts the video recognition start time in response to a received instruction to adjust the video recognition start time to add the pre-swing to the recognition; and further adjusts the recognition frequency to achieve adjustment of the recognition accuracy.

[0080] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0081] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0082] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0083] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0084] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0085] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A video processing method, characterized in that: include: Receiving a video recognition request sent by the first platform, the video recognition request including a time range, a recognition frequency, and a target tag for video recognition, the recognition frequency being X frames per second, wherein the first platform sends the video recognition request in response to the received recording start signal; In response to the video recognition request, retrieve and recognize the corresponding video to generate recognition data, and send the recognition data to the first platform, wherein the recognition data includes a plurality of tags; Receiving a video review request sent by the first platform, wherein the first platform confirms whether the target tag is included in the received multiple tags, and if so, sends a video review request after the first preset time is reached or a recording end signal is received, whichever is earlier; In response to a received video review request, if the trigger condition of the video review request is reaching a first preset duration, the time range of video recognition is adjusted according to the first preset duration, and subsequent videos are divided according to the duration of the adjusted video recognition time range as the time range for the next video recognition until the event is complete; and / or if the trigger condition of the video review request is receiving a recording end signal, confirming the duration of the complete event of the target tag, if the duration is not greater than a second preset duration, the time range of video recognition is not adjusted; if the duration is greater than the second preset duration, the time range of video recognition is adjusted according to the duration of the complete event of the target tag.

2. The video processing method according to claim 1, characterized in that: If the trigger condition of the video review request is reaching the first preset duration, an abnormal instruction is reported.

3. The video processing method according to claim 1, characterized in that: In response to receiving an instruction to adjust the start time of video recognition, the start time of the corresponding video recognition time range is moved forward to the adjusted start time; wherein, the instruction to adjust the start time of video recognition refers to receiving an execution instruction within a third preset time period before the start time of an event of a target tag, and the video recognition start time is confirmed according to the time when the execution instruction is received.

4. The video processing method according to claim 3, characterized in that: The receiving of the execution instruction within a third preset time period before the event start time of a target tag, and confirming the video recognition start time according to the time when the execution instruction is received, comprises: The third preset time length is divided into n equal parts, and whether an execution instruction is received in a corresponding time period is detected one by one. If an execution instruction is received, the start time of the time period in which the execution instruction is received is used as the video recognition start time.

5. The video processing method according to claim 1 or 3, characterized in that: In response to a received video review request and / or in response to a received instruction to adjust a video recognition start time, the recognition frequency is adjusted.

6. The video processing method according to claim 5, characterized in that: According to the adjusted time range and recognition frequency of the video recognition, the corresponding video is retrieved and recognized to generate adjusted recognition data, and the adjusted recognition data is sent to the first platform.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that enables a processor to execute the method according to any one of claims 1 to 6.