Video processing method and related device

Abstract

The embodiment of the invention provides a video processing method and a related device, and the video processing method comprises the steps: obtaining an active region video of an object, determining a video collection strategy corresponding to the active region video, reading prompt information of the video collection strategy, carrying out the video coding of the active region video through a video coder, obtaining video features, and outputting the video features. And the information encoder carries out information encoding on the prompt information to obtain information features, inputs the video features and the information features into a model to carry out biological behavior identification to obtain an identification result, and generates a traceability video of the object based on the video of the active area under the condition that the identification result meets a traceability condition corresponding to a video acquisition strategy. According to the embodiment of the invention, the effectiveness of the generated traceability video of the object can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a video processing method and related devices. Background Art

[0002] With the continuous development and progress of the Internet and information technology, more and more services are provided online. With the improvement of living standards, users have higher and higher requirements for the items or food they use in daily life. Based on this, in order to enhance users' perception of the items or food they use, traceability has become an important way for providers or recommenders to recommend items or food to users. How to improve the convenience and efficiency of traceability has become a focus of increasing concern for providers, recommenders and users. Summary of the invention

[0003] In a first aspect, an embodiment of the present application provides a video processing method, including: Acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; The video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features; Inputting the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result; When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

[0004] It can be seen that in the embodiment of the present application, during the video processing, after obtaining the activity area video of the object, the video acquisition strategy corresponding to the activity area video is first determined, and then the prompt information of the video acquisition strategy is read, so that the activity area video is corresponded to the video acquisition strategy, and the subsequent biological behavior recognition is performed through the prompt information of the video acquisition strategy, thereby improving the flexibility and effectiveness of the biological behavior recognition performed on the activity area video. On this basis, the video encoder encodes the activity area video to obtain video features, and the information encoder encodes the prompt information to obtain information features, and then the video features and the information features are input into the model for biological behavior recognition to obtain recognition. The results are obtained by distinguishing the biological behaviors. Therefore, before the model is used for biological behavior recognition, the activity area video and prompt information are encoded respectively to obtain accurate encoding results for the activity area video and prompt information. The video features and information features obtained by encoding are then used as input to enable the model to generate behavior recognition, thereby improving the accuracy of the model's biological behavior recognition. With the help of the model, the biological behavior corresponding to the video acquisition strategy is recognized on the activity area video corresponding to the video acquisition strategy. After obtaining the recognition result, the tracing condition corresponding to the video acquisition strategy is used to affect the generation of the tracing video. Therefore, the video acquisition strategy is used as the basis for biological behavior recognition and tracing video generation, thereby improving the richness and effectiveness of the generated tracing video.

[0005] In a second aspect, an embodiment of the present application provides a video processing device, including: A video acquisition module, used to acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; An encoding module, wherein a video encoder performs video encoding on the active area video to obtain video features, and an information encoder performs information encoding on the prompt information to obtain information features; A biological behavior recognition module, used for inputting the video features and the information features into a model to perform biological behavior recognition and obtain a recognition result; In the case where the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a video generation module is run, and the video generation module is used to generate a tracing video of the object based on the activity area video.

[0006] In a third aspect, an embodiment of the present application provides a video processing device, comprising: a processor; and a memory configured to store computer executable instructions, wherein the computer executable instructions, when executed, cause the processor to execute the video processing method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer-executable instructions, which, when executed by a processor, implement the video processing method as described in the first aspect.

[0008] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the video processing method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. Figure 1 A schematic diagram of an implementation environment of a video processing method provided in an embodiment of the present application; Figure 2 A processing flow chart of a video processing method provided in an embodiment of the present application; Figure 3 A schematic diagram of a video encoding process provided in an embodiment of the present application; Figure 4 A text encoding processing schematic diagram provided in an embodiment of the present application; Figure 5 A schematic diagram of a process for encoding and biometric behavior recognition provided in an embodiment of the present application; Figure 6 A processing flow chart of a video processing method for generating a tracing video of a captive animal provided in an embodiment of the present application; Figure 7 A processing flow chart of another video processing method for generating a tracing video of a domestic animal provided in an embodiment of the present application; Figure 8 A schematic diagram of a video processing device provided in an embodiment of the present application; Fig. 9 A schematic diagram of the structure of a video processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0011] In practical applications, in order to improve the perception of items or food, the production process of items or the breeding process of ingredients can be traced to provide safer and healthier items or food; In order to achieve convenient and effective traceability, an embodiment of the present application provides a video processing method, which performs biological behavior recognition and generates a traceability video on the activity area video of the object, so that the traceability video of the object can be queried during the traceability process, thereby achieving fast and true object traceability; in the specific process of performing biological behavior recognition and generating a traceability video on the activity area video of the object, after obtaining the activity area video of the object, first read the prompt information of the video acquisition strategy corresponding to the activity area video, then encode the activity area video through a video encoder to obtain video features, and encode the prompt information through an information encoder to obtain information features, and then input the video features and the information features into a model for biological behavior recognition to obtain a recognition result, and finally, when the recognition result meets the traceability condition corresponding to the video acquisition strategy, generate the object's traceability video based on the activity area video, thereby performing biological behavior recognition and traceability video generation on the activity area video according to the video acquisition strategy, improving the flexibility and richness of processing, and performing biological behavior recognition with the help of a model to improve the accuracy and effectiveness of biological behavior recognition.

[0012] The video processing method provided in one or more embodiments of this specification can be applied to the implementation environment of object tracing video processing, such as Figure 1 As shown, the implementation environment includes at least a server 101, and the server 101 may be one or more servers, a server cluster consisting of several servers, or a cloud server of a cloud computing platform; In addition, the implementation environment may also include a video encoder 102, an information encoder 103 and a model 104, wherein the video encoder 102 is used for video encoding, the information encoder 103 is used for information encoding, and the model 104 is used for biological behavior recognition; The implementation environment may also include a video acquisition component 105, which may be a camera that can be zoomed and rotated, and the video acquisition component may also support configuration of video acquisition strategies; the video acquisition component 105 may be configured in the activity area of ​​the object to collect video of the activity area of ​​the object; In this implementation environment, the video acquisition component 105 acquires the activity area video of the object. After the server 101 obtains the activity area video of the object acquired by the video acquisition component 105, it first determines the video acquisition strategy corresponding to the activity area video and reads the prompt information of the video acquisition strategy. Then, the video encoder 102 encodes the activity area video to obtain video features, and the information encoder 103 encodes the prompt information to obtain information features. The video features and the information features are then input into the model 104 for biological behavior recognition to obtain recognition results. Finally, when the recognition results meet the traceability conditions corresponding to the video acquisition strategy, the traceability video of the object is generated based on the activity area video.

[0013] Figure 2 A processing flow chart of a video processing method provided in this embodiment, refer to Figure 2 The video processing method provided in this embodiment specifically includes steps S202 to S208: Step S202: acquiring the activity area video of the object, determining the video acquisition strategy corresponding to the activity area video, and reading the prompt information of the video acquisition strategy.

[0014] The object in this embodiment can be a product obtained by production or a captive; the captive can be a collectively raised organism, such as a free-range chicken. For the captive object, generally speaking, an activity area is set for the captive so that the captive can move freely. The activity area video in this embodiment can refer to a video shot for the activity area of ​​the captive; for the object of the object, generally speaking, a production area of ​​the object is set to produce the object. The activity area video in this embodiment can refer to a video shot for the production area of ​​the object.

[0015] For objects, whether they are produced manually or by machines, there are certain production times and rest times. For domestic animals, there are certain activity times and certain rest times. In order to avoid the video acquisition components deployed in the activity area from continuously acquiring videos, resulting in invalidity of the activity area videos acquired during a long rest time and loss of the video acquisition components, in this embodiment, the video acquisition strategy can be configured so that the video acquisition components acquire videos according to the video acquisition strategy, thereby improving the availability of the acquired activity area videos and saving the video acquisition components and video storage space. This embodiment can configure the video acquisition strategy according to different traceability requirements.

[0016] Taking free-range chickens as an example, in the process of configuring the video acquisition strategy, you can configure the video acquisition strategy for chickens leaving the chicken coop: continuous video acquisition from 7 am to 10 am, and detection at every hour; or you can also configure the video acquisition strategy for chickens returning to the chicken coop: continuous video acquisition from 4 pm to 7 pm, and detection at every hour; or you can also configure the close-up video acquisition strategy: 5 minutes of close-up video acquisition every hour from 7:30 am to 7:30 pm. In addition to configuring the acquisition time and acquisition duration, the video acquisition strategy can also configure prompt information. For example, the prompt information of the video acquisition strategy for chickens leaving the chicken coop can be configured as: whether there is a chicken leaving the chicken coop; the prompt information of the video acquisition strategy for chickens returning to the chicken coop can be configured as: whether there is a chicken returning to the chicken coop; or the prompt information for the video acquisition strategy for chickens leaving the chicken coop and the video acquisition strategy for chickens returning to the chicken coop can be: whether there is a chicken currently; the prompt information of the close-up video acquisition strategy can be configured as: whether there is a chicken currently, and if so, summarize the interesting behavior of the chicken. It should be noted that the above-mentioned video acquisition strategy for chickens leaving the chicken coop, the video acquisition strategy for chickens returning to the chicken coop, and the close-up video acquisition strategy are only exemplary. In actual applications, the video acquisition strategy may include any one or more of the above three video acquisition strategies, and may also include other configured video acquisition strategies, which are not limited in this embodiment. Optionally, the video acquisition strategy includes video acquisition time, video acquisition duration, prompt information and / or traceability conditions. It should also be noted that the configuration of the video acquisition strategy for objects is similar to the above, and can also be configured according to actual needs, which will not be repeated in this embodiment.

[0017] In specific implementation, after obtaining the activity area video of the object captured by the video capture component, first determine the video capture strategy corresponding to the activity area video based on the capture time and / or capture duration of the activity area video, and then read the prompt information of the video capture strategy.

[0018] Step S204: the video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features.

[0019] In this embodiment, after obtaining the activity area video and prompt information, biological behavior recognition is performed based on the activity area video and prompt information. In order to improve the accuracy and effectiveness of biological behavior recognition using a large language model, in this step, the activity area video is first encoded to obtain video features, and the prompt information is encoded to obtain information features.

[0020] In specific implementation, due to the difference between video and information, in order to ensure the effectiveness of encoding, in an optional implementation provided by this embodiment, the video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features, that is, the active area video is input into the video encoder for video encoding to obtain video features, and the prompt information is input into the information encoder for information encoding to obtain information features. The following specifically describes the process of the video encoder performing video encoding on the active area video to obtain video features and the process of the information encoder performing information encoding on the prompt information to obtain information features.

[0021] (1) The video encoder encodes the active area video to obtain video features; During specific implementation, the video encoder performs video encoding on the active area video. In the process of obtaining the video features, the active area video needs to be input into the video encoder so that the video encoder performs video encoding to obtain the video features, that is, the active area video is input into the video encoder for video encoding; in the specific implementation process, in order to improve the encoding efficiency of video encoding while ensuring the validity of the video, in an optional implementation manner provided by this embodiment, in the process of inputting the active area video into the video encoder for video encoding to obtain the video features, the following operations are performed: Extracting video frames from the active area video to obtain a video frame sequence; The video frame sequence is input into the video encoder for video encoding to obtain the video features.

[0022] Specifically, the active area video may be frame skipped to obtain a video frame sequence, and the video frame sequence may be input into a video encoder for video encoding to obtain video features.

[0023] In the specific execution process of video encoding by the video encoder, in order to pay more attention to the continuous information of the video, each video frame in the video frame sequence can be encoded as an independent video feature; but since the size of the video frame is h*w*c, where h, w, c represent the height, width and number of channels of the video frame respectively; the size of the video frame is huge, the computational complexity of directly inputting the video frame into the embedding layer for video frame transformation is large. Therefore, in the process of video encoding the video frame sequence, the video encoder can first perform dimensionality reduction processing on each video frame to obtain a reduced-dimensionality video frame sequence, and then perform video encoding based on the reduced-dimensionality video frame sequence.

[0024] In an optional implementation manner provided by this embodiment, the video encoder may perform video encoding in the following manner: Position encoding is performed on a video frame sequence to obtain position features, and dimension reduction is performed on each video frame in the video frame sequence to obtain multiple dimension-reduced video frames, and video frame encoding is performed on each dimension-reduced video frame to obtain initial video frame features; Performing feature fusion on the position features of each video frame and the features of the initial video frame to obtain fusion features of each video frame; The fusion features of the video frames are input into an autoregressive model to perform autoregressive calculation, and the video frame codes of the video frames are obtained as the video codes.

[0025] Specifically, position coding is introduced to enable one-to-one matching of video frames and prompt information.

[0026] For example, Figure 3 As shown, the video encoder includes a dimensionality reduction model (dimensionality reduction module), a video frame encoding model (embedding layer), a position encoding model (position encoding module), a feature fusion model (feature fusion module) and an autoregressive model (autoregressive module); wherein the input video frame sequence can be ; Where n is the number of video frames in the video frame sequence; Then, on the one hand, the dimensionality reduction model can use the SVD (Singular Value Decomposition) dimensionality reduction method to reduce the dimension of each video frame into a t*c (t<(h*w)) matrix. After the dimensionality reduction process , the reduced-dimensionality video frame after dimensionality reduction processing Input video frame coding model, that is, the embedding layer performs video frame coding operation and outputs initial video features Matrix of On the other hand, the position coding model performs position coding on the video frame sequence. In the process of position coding, first define:

[0027]

[0028] Among them, PE is the position encoding matrix initialized with all 0s, expressed as the maximum position matrix, and P0 is the normalized set of 1 to n; In order to allow PE to have position encoding information, the following calculation is performed:

[0029]

[0030]

[0031]

[0032] Among them, div represents the coefficient of variation, norm is the normalization function, and then the corresponding inverse trigonometric functions are assigned to the odd columns PE (:, 0::2) and even columns PE (:, 1::2) of the PE matrix to form the corresponding encoding tensor, and the dimension of PE becomes PE ; After obtaining the initial video features After adding the position feature PE, the feature fusion model superimposes the position feature and the initial video feature to obtain the fusion feature X1= ; Among them, X1 ; After obtaining X1, X1 is input into the autoregressive model for autoregressive calculation. The autoregressive calculation is as follows:

[0033]

[0034]

[0035] Further calculate the score matrix:

[0036] Reuse

[0037] Calculate video features It should be noted that the calculated .

[0038] (2) The information encoder encodes the prompt information to obtain information features; The prompt information may be a prompt text, and correspondingly, the information encoder may also be a text encoder.

[0039] In the specific implementation process, position coding is also introduced in the process of information coding to improve the effectiveness of information coding. In an optional implementation manner provided by this embodiment, the information encoder can encode the prompt information in the following manner: Performing position encoding on the prompt information to obtain position features, and performing initial information encoding on the prompt information to obtain initial information features; The initial information feature and the position feature are subjected to feature fusion, and the fused feature obtained by the feature fusion is input into an autoregressive model for autoregressive calculation to obtain the information code.

[0040] For example, Figure 4 As shown, the text encoder includes a text encoding model (embedding layer), a position encoding model (position encoding module), a feature fusion model (feature fusion module) and an autoregressive model (autoregressive module). First, the prompt text is obtained. , where N is 1 and n is the number of prompt texts. First, the prompt text X is input into the text encoding model, that is, the embedding layer performs text conversion to obtain the initial text features. , where 512 is the embedding dimension; In order to make the text vector have richer position information, position information is introduced:

[0041]

[0042] Among them, PE is the position encoding matrix initialized with all 0s, expressed as the maximum position matrix, and P0 is the normalized set of 1 to n; In order to allow PE to have position encoding information, the following calculation is performed:

[0043]

[0044]

[0045]

[0046] Among them, div represents the coefficient of variation, norm is the normalization function, and then the corresponding inverse trigonometric functions are assigned to the odd columns PE (:, 0::2) and even columns PE (:, 1::2) of the PE matrix to form the corresponding encoding tensor, and the dimension of PE becomes PE ; After obtaining the initial text features After adding the position feature PE, the feature fusion model superimposes the position feature and the initial video feature to obtain X1= ; Among them, X1 ; After obtaining X1, X1 is input into the autoregressive model for autoregressive calculation. The autoregressive calculation is as follows:

[0047]

[0048]

[0049] Further calculate the score matrix:

[0050] Reuse

[0051] Calculate video features It should be noted that the calculated .

[0052] The above specifically describes the video encoding process of the video encoder and the information encoding process of the information encoder. In order to improve the encoding accuracy of the video encoder and the information encoder obtained by training, in this embodiment, the video encoder and the information encoder can be trained together. During the training process, when the training is initialized, define = , as the model is continuously trained, As the model parameters are continuously updated, finally through As a loss function, the loss is calculated; specifically, in an optional implementation manner provided in this embodiment, the video encoder and the information encoder are trained in the following manner: Read the active area video sample and the corresponding prompt information sample; Inputting the video frame sequence corresponding to the active area video sample into the video encoder to be trained for video encoding to obtain sample video features, and inputting the prompt information sample into the information encoder to be trained for information encoding to obtain sample information features; The training loss is calculated according to the sample video features and the sample information features, and the parameters of the video encoder to be trained and the information encoder to be trained are updated based on the training loss.

[0053] Specifically, the video encoder to be trained and the information encoder to be trained are trained in the above manner until the encoders converge or reach a preset number of training times, thereby obtaining a video encoder and an information encoder.

[0054] Step S206: input the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result.

[0055] In the above steps, the video encoder encodes the active area video to obtain video features, and the information encoder encodes the prompt information to obtain information features. On this basis, in this step, the video features and information features are input into the model for biological behavior recognition to obtain recognition results. The model can be a large language model, such as LLM (Large Language Model). For example, Figure 5 As shown, the prompt text is input into a text encoder for text encoding to obtain text features, a video frame sequence consisting of at least one video frame is input into a video encoder for video encoding to obtain video features, and then the text features and video features are input into a model (such as LLM) for biological behavior recognition to obtain recognition results; wherein, in order to define the structure and shape of the output, placeholders can also be input into the model.

[0056] In specific implementation, due to different prompt information, the recognition results of biological behavior recognition are also different; that is, due to different video acquisition strategies, the corresponding recognition results are also different. Taking the video acquisition strategy of chickens leaving the chicken coop as an example, since the prompt information is: whether the behavior of leaving the chicken coop exists; then the corresponding recognition result is yes or no; taking the close-up video acquisition strategy as an example, since the prompt information is: whether there are chickens at present, if so, summarize the interesting behaviors of chickens; then the corresponding recognition result is yes, the interesting behaviors of chickens are chasing or the chickens do not exist. That is, the large language model uses different methods to perform biological behavior recognition under different prompt information, obtains recognition results and outputs them. In an optional implementation manner provided in this embodiment, biological behavior recognition can be implemented in the following manner: Perform semantic recognition according to the information features to obtain a semantic recognition result; According to the semantic recognition result, a recognition result matching the video feature is generated.

[0057] Specifically, in the process of biological behavior recognition, the model performs semantic recognition based on information features, and after obtaining the semantic recognition results, it can generate recognition results that match the video features according to the processing strategy corresponding to the semantic recognition results; for example, if the semantic recognition result is that the chicken leaves the chicken coop, then the video features are used to calculate whether it contains free-range chickens, and if so, the recognition result is determined to be yes; if not, the recognition result is determined to be no; or, if the semantic recognition result is that the chicken returns to the chicken coop, then the video features are used to calculate whether it contains free-range chickens, and if so, the recognition result is determined to be no; if not, the recognition result is determined to be yes; or, if the semantic recognition result is an interesting behavior, then the video features are used to calculate whether it contains free-range chickens, and if not, the recognition result is determined to be no, and if so, the interesting behavior is calculated based on the video features, and interesting behaviors such as jumping and / or running are output. In addition, in addition to interesting behaviors, the recognition results may also include the time of occurrence of interesting behaviors; correspondingly, the video description information of the generated tracing video may also include the time of occurrence and interesting behaviors.

[0058] Step S208: When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

[0059] The traceability video includes a video recording the behavior of the captive animal or a video recording the production progress of the item.

[0060] In the specific implementation, the video features and information features are input into the model for biological behavior recognition. After the recognition result is obtained, it is necessary to detect whether the recognition result meets the traceability condition corresponding to the video acquisition strategy. Due to the different video acquisition strategies, the corresponding traceability conditions are also different, and the specific detection methods are also different. Furthermore, the process of generating the traceability video of the object based on the activity area video is also different. Let's continue to take the chicken leaving the chicken coop video acquisition strategy, the close-up video acquisition strategy and the chicken returning to the chicken coop video acquisition strategy as examples; among them, the essence of the chicken leaving the chicken coop video acquisition strategy and the chicken returning to the chicken coop video acquisition strategy is to obtain relevant videos of the chicken leaving the chicken coop and the chicken returning to the chicken coop. The large language model can be used to identify whether there are chickens leaving the chicken coop or chickens returning to the chicken coop in the collected activity area videos. Related videos of chickens returning to the chicken coop, so the video collection strategy of chickens leaving the chicken coop and the video collection strategy of chickens returning to the chicken coop can be regarded as the same type of video collection strategy for judging the traceability conditions; the essence of the close-up video collection strategy is to obtain the interesting behaviors of free-range chickens, and the large language model can be used to identify the interesting behaviors contained in the activity area video. Therefore, the close-up video collection strategy can be regarded as a separate type for judging the traceability conditions; taking the video collection strategy of chickens leaving the chicken coop and the video collection strategy of chickens returning to the chicken coop as the first video collection strategy, and taking the close-up video collection strategy as the second video collection strategy as an example, the process of detecting whether the recognition result meets the traceability conditions corresponding to the video collection strategy and the process of generating the traceability video are specifically explained.

[0061] (1) First video acquisition strategy In order to efficiently and accurately obtain the tracing video corresponding to the first video acquisition strategy, in an optional implementation manner provided in this embodiment, when the video acquisition strategy corresponding to the activity area video is the first video acquisition strategy, the following method can be used to detect whether the recognition result meets the tracing condition corresponding to the video acquisition strategy: Reading historical recognition results of the activity area video associated with the activity area video; In the case where the historical recognition result is inconsistent with the recognition result, determining that the recognition result satisfies the traceability condition; When the historical recognition result is consistent with the recognition result, it is determined that the recognition result does not meet the traceability condition and no processing is performed.

[0062] Optionally, the associated activity area video includes an activity area video collected at a second collection time associated with the first collection time determined according to the first collection time of the activity area video; for example, if the collection time of the activity area video is 8 o'clock in the morning, then the associated activity area video is the activity area video collected at 7 o'clock in the morning. That is, the associated activity area video may be an activity area video collected at a previous collection time corresponding to the collection time of the activity area video. For example, the associated collection time of the collection time of 8 o'clock in the morning is 7 o'clock in the morning; the associated collection time of the collection time of 7:50 in the morning is 7:40 in the morning. It should be noted that the previous collection time corresponding to the collection time of the activity area video includes the previous collection time of the collection time of the activity area video in the corresponding video collection strategy.

[0063] Specifically, when the biological behavior recognition result of the activity area video is inconsistent with the biological behavior recognition result of the activity area video associated with the activity area video, it is determined that the recognition result meets the traceability condition corresponding to the video acquisition strategy.

[0064] For example, the video acquisition strategy corresponding to the activity area video is the chicken leaving the chicken coop video acquisition strategy, and the prompt text is whether there are chickens at present; after the activity area video is subjected to biological behavior recognition in the above manner, the recognition result is yes, then the associated second acquisition time is determined to be 7 o'clock in the morning according to the first acquisition time of the activity area video at 8 o'clock in the morning, and the activity area video at 7 o'clock in the morning is read as the associated activity area video, and the recognition result obtained after the biological behavior recognition of the associated activity area video is read is no, then there are chickens in the activity area video at 8 o'clock in the morning, and there are no chickens in the activity area video at 7 o'clock in the morning, which means that there is a behavior of chickens leaving the chicken coop to the activity area between 7 o'clock and 8 o'clock in the morning, and it is determined that the recognition result of the activity area video meets the traceability condition; it should be noted that if the recognition result obtained after the biological behavior recognition of the associated activity area video is yes, it means that the chickens have been in the activity area between 7 o'clock and 8 o'clock in the morning, and there is no behavior of chickens leaving the chicken coop to the activity area, therefore, it is determined that the recognition result of the activity area video does not meet the traceability condition.

[0065] Further, when the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, the tracing video of the object is generated based on the activity area video. When the video acquisition strategy is the first video acquisition strategy, in order to ensure the validity of the generated tracing video, in an optional implementation manner provided by this embodiment, the process of generating the tracing video of the object based on the activity area video can be implemented in the following manner: Reading a first acquisition time of the activity area video and a second acquisition time of the associated activity area video, and determining a tracing time interval based on the first acquisition time and the second acquisition time; The activity area video collected within the tracing time interval is determined as the tracing video.

[0066] Specifically, since the associated activity area video is the activity area video collected at the previous collection time corresponding to the activity area video, the activity area video collected within the tracing time interval from the second collection time to the first collection time is determined as the tracing video; in other words, the activity area video collected within the tracing time interval constituted by the collection time of the associated activity area video of the activity area video to the collection time of the activity area video is determined as the tracing video.

[0067] Continuing with the above example, the activity area video collected between 7 and 8 in the morning is used as the source video under the chicken leaving the chicken coop video collection strategy.

[0068] That is, step S208 can also be replaced by, when the recognition result meets the tracing condition corresponding to the video acquisition strategy, determining the tracing time interval based on the first acquisition time of the activity area video and the associated second acquisition time, and using the activity area video acquired within the tracing time interval as the tracing video of the object, and one or more other steps provided in this embodiment can be selected to form a new implementation method.

[0069] In order to further improve the effectiveness of tracing, and avoid using long-term activity area videos as tracing videos, which will result in watching a large number of invalid videos during the tracing process, the divide-and-conquer method can be used to generate tracing videos. Specifically, in an optional implementation provided by this embodiment, when the recognition result is inconsistent with the historical recognition result, the following method can be used to determine whether the recognition result meets the tracing condition: Reading a time index of a first acquisition time of the activity area video, and detecting whether the time index is a preset index; If so, determining that the identification result satisfies the traceability condition; If not, split the associated time interval to obtain at least one sub-time; A tracing video of the object is generated according to the recognition results corresponding to the activity area videos at each sub-time.

[0070] Optionally, the associated time interval is determined based on the first collection time and the second collection time.

[0071] Specifically, when the recognition results of the activity area video and the associated activity area video are inconsistent, if the time index of the acquisition time of the activity area video is a preset index, it is determined that the recognition result meets the tracing condition, and then the associated time interval from the second acquisition time (associated acquisition time) to the first acquisition time (acquisition time) is used as the tracing time interval, and the activity area video collected within the tracing time interval is used as the tracing video; if the time index of the acquisition time of the activity area video is not a preset threshold, the associated time interval is split into sub-times to obtain at least one sub-time, and based on the recognition results of the activity area videos of adjacent sub-times in the at least one sub-time, it is detected whether the recognition results of the adjacent sub-times meet the corresponding tracing conditions, and then the tracing video of the object is generated based on the activity area videos of the adjacent sub-times.

[0072] It should be noted that the above process can be carried out in a loop until the time index of the sub-time is the preset index; specifically, if the time index of the acquisition time of the activity area video is not the preset threshold, the associated time interval is split into sub-times to obtain at least one sub-time, and the recognition results of the activity area video of each sub-time are detected to be consistent with the recognition results of the associated activity area video. If so, no processing is performed; if not, the time index of the first sub-time (the acquisition time of the activity area video that is inconsistent with the recognition result of the associated activity area video) is detected to be the preset index. If so, the associated time interval from the second sub-time (the previous sub-time of the first sub-time) to the first sub-time is used as the tracing time interval and the activity area video collected in the tracing time interval is used as the tracing video; if not, the associated time interval from the second sub-time to the first sub-time is further split into sub-times, and the above process is repeated. It should be noted that in the process of sub-time splitting, the sub-time splitting can be performed according to the priority of 30 minutes, 10 minutes, 5 minutes, and 1 minute, and the preset index can be the index corresponding to 1 minute.

[0073] In addition to ending the divide-and-conquer method according to preset indicators, the divide-and-conquer method can also be ended by using the interval duration of the associated time interval; for example, when the recognition result of the activity area video is inconsistent with the recognition result of the associated activity area video, detect whether the interval duration of the associated time interval formed by the second acquisition time and the first acquisition time is the preset duration, such as 10 minutes; if not, split the associated time interval into sub-times, and if so, determine that the recognition result meets the traceability conditions.

[0074] Continuing with the example of the chickens going out of the chicken coop to the activity area between 7 and 8 in the morning, when it is detected that the chickens go out of the chicken coop to the activity area between 7 and 8 in the morning, that is, the recognition result at 7 in the morning is no, and the recognition result at 8 in the morning is yes, since the duration of the associated time interval from 7 to 8 in the morning is 1 hour, if the chickens go out of the chicken coop to the activity area at 7:50, the activity area video from 7 to 7:50 included in the traceability video is empty. Directly using the activity area video from 7 to 8 as the traceability video will result in watching the empty activity area video for a long time, affecting In order to improve the tracing experience, the time from 7 to 8 o'clock is divided into sub-times. First, the sub-times are divided into 30 minutes, namely 7 o'clock, 7:30 and 8 o'clock. The recognition results of the activity area video at 7 o'clock (which can be the video collected within one minute at 7 o'clock) and the recognition results of the activity area video at 7:30 are respectively detected to see whether they are consistent, and the recognition results of the activity area video at 7:30 are detected to see whether they are consistent with the recognition results of the activity area video at 8 o'clock. After detection, if the recognition result of the activity area video at 7 o'clock is consistent with the recognition result of the activity area video at 7:30, it is determined that the tracing condition is not met and no processing is performed. Yes, the recognition result of the activity area video at 7:30 is inconsistent with the recognition result of the activity area video at 8 o'clock, then the interval length of the associated time interval from 7:30 to 8 o'clock is detected to see whether it is the preset duration of 10 minutes. After the detection interval length is not equal to the preset duration, the associated time interval from 7:30 to 8 o'clock is split into sub-times of 10 minutes to obtain 7:30, 7:40, 7:50 and 8 o'clock, among which the second collection time associated with 7:30 is empty, the second collection time associated with 7:40 is 7:30, the second collection time associated with 7:50 is 7:40, and the second collection time associated with 8 o'clock is 7:40. The collection time is 7:50, and the recognition results of the activity area video at each collection time are detected to be consistent with the associated second collection time. If so, it is determined that the traceability conditions are not met. After detection, the recognition result of the activity area video at 8 o'clock is inconsistent with the recognition result of the activity area video at the associated collection time of 7:50, and the interval duration of the associated time interval from 7:50 to 8 o'clock is the preset duration of 10 minutes. The associated time interval from 7:50 to 8 o'clock is used as the traceability time interval, and the activity area video collected within the traceability time interval from 7:50 to 8 o'clock is used as a traceability video corresponding to the video collection strategy of chickens leaving the chicken coop.

[0075] In addition, when the prompt information of the chicken leaving the chicken coop video collection strategy is whether the chicken leaving the chicken coop behavior exists, if the recognition result is yes, it is determined that the corresponding traceability condition is met, and if the recognition result is no, it is determined that the corresponding traceability condition is not met. That is, step S208 can also be replaced by, when the recognition result is a positive recognition result, generating a traceability video of the object based on the recognition area video, and one or more other steps provided in this embodiment can be selected to form a new implementation method.

[0076] (2) Second video acquisition strategy In another optional implementation provided by this embodiment, the following method can be used to detect whether the recognition result meets the tracing condition corresponding to the video acquisition strategy: when the recognition result is not empty, it is determined that the recognition result meets the tracing condition; when the recognition condition is empty, it is determined that the recognition result does not meet the tracing condition; correspondingly, when the recognition result is not empty, the activity area video is determined as the tracing video.

[0077] It should be noted that it is also possible not to divide different types of video acquisition strategies, and to detect whether the traceability condition is met according to any of the above two, which is not limited in this embodiment. In addition, step S208 can also be replaced by, when the recognition result meets the traceability condition corresponding to the video acquisition strategy, using the activity area video as the traceability video of the object, and one or more other steps provided in this embodiment can be selected to form a new implementation method.

[0078] During the specific execution process, after generating the traceability video of the object, video description information of the traceability video can also be generated according to the video acquisition strategy, recognition results and / or video information of the traceability video (such as video time); for example, a traceability video of a chicken leaving the chicken coop at 7:50 in the morning on x-month-x-year, or a traceability video of a chicken jumping at t-hour on x-month-x-year.

[0079] After the object's source tracing video is generated, the object's source tracing video can be stored to trace the object's source; it should be noted that one video acquisition strategy can correspond to multiple source tracing videos. In an optional implementation provided in this embodiment, the object's source tracing can be performed in the following manner: According to the user's request for tracing the object, the tracing video of the object is read and sent to the user.

[0080] It should also be noted that the video processing method provided in this embodiment can be performed according to the configuration of the video acquisition strategy. For example, the video acquisition strategy for chickens leaving the chicken coop is configured to perform detection at every hour, and the above process is performed at every hour. In addition, the video acquisition strategy for chickens leaving the chicken coop can also be configured to perform detection at ten o'clock in the morning, and detection is performed starting from the activity area video at every hour at ten o'clock in the morning. This embodiment is not limited here.

[0081] In summary, the video processing method provided in this embodiment configures a video acquisition component in the activity area of ​​the object and configures a video acquisition strategy so that the video acquisition component performs video acquisition according to the video acquisition strategy. There is no need to continuously perform video acquisition, thereby avoiding the loss of the video acquisition component caused by the acquisition of invalid videos. After acquiring the activity area video of the object acquired by the video acquisition component, the corresponding video acquisition strategy is determined according to the acquisition time and / or acquisition duration of the activity area video, and the prompt information of the video acquisition strategy is read, and then the video is encoded by the video encoder to obtain the video features, and the prompt information is encoded by the information encoder to obtain the information features, and then the video features and the information features are input into the large language model for biological behavior recognition to obtain the recognition result. In this way, the large language model is introduced to improve the convenience and efficiency of biological behavior recognition. When the recognition result meets the tracing condition corresponding to the video acquisition strategy, the tracing video of the object is generated based on the activity area video, and the tracing video and video description information are stored, so as to send the tracing video and video description information to the user in the subsequent object tracing process, thereby improving the effectiveness and convenience of tracing and improving the user's perception of the object.

[0082] The following takes the application of a video processing method provided by this embodiment in the scene of generating a tracing video of a domestic animal as an example, combined with Figure 6 The video processing method provided in this embodiment for the scene of generating a tracing video of a domestic animal is further described. Figure 6 The video processing method applied to the scene of tracing video generation of domesticated animals specifically includes the following steps.

[0083] Step S602: obtaining a video of the activity area captured by a camera configured in the activity area of ​​the animal.

[0084] Step S604, determining a corresponding video acquisition strategy according to the acquisition time of the activity area video, and reading a prompt text of the video acquisition strategy.

[0085] Step S606: perform frame skipping on the active area video to obtain a video frame sequence.

[0086] Step S608: the video encoder performs video encoding on the video frame sequence to obtain video features.

[0087] Step S610: the text encoder performs text encoding on the prompt text to obtain text features.

[0088] The execution order of step S610 and steps S606 to S608 is not limited here.

[0089] Step S612: Input the video features and text features into the model for biological behavior recognition to obtain recognition results.

[0090] Step S614, determining the adjacent activity area videos according to the acquisition time of the activity area video.

[0091] Optionally, the adjacent activity area video of the activity area video includes an activity area video acquired at a previous acquisition time corresponding to the acquisition time of the activity area video.

[0092] Step S616, detecting whether the recognition result of the activity area video is consistent with the recognition result of the adjacent activity area video; If so, no action is required; If not, execute step S618.

[0093] Step S618, detecting whether the duration of the associated time interval from the second acquisition time of the adjacent activity area video to the first acquisition time of the activity area video is a preset duration; If so, execute step S622.

[0094] If not, execute step S620.

[0095] Step S620, split the associated time interval into sub-times to obtain at least one sub-time, and return to execute steps S614 to S616 based on the activity area video of each sub-time.

[0096] Step S622: Use the activity area video collected within the associated time interval as the source tracing video of the captive under the video collection strategy.

[0097] Thereafter, video description information may be generated based on the video time and video acquisition strategy of the source tracing video, and the source tracing video and the video description information may be associated and stored.

[0098] In addition, steps S602 to S622 may also be replaced by: when it is detected that the detection condition of the video acquisition strategy is triggered, obtaining the activity area video of the captive at at least one acquisition time corresponding to the video acquisition strategy; the video encoder performs video encoding on the video frame sequence corresponding to each activity area video to obtain video features, and the text encoder performs text encoding on the prompt text of the video acquisition strategy to obtain text features, and the text features and the video features of each activity area video are input into a large language model for biological behavior recognition to obtain the recognition results of each activity area video; the adjacent activity area videos of each activity area video are determined according to the acquisition time of each activity area video; at least one target activity area video whose recognition result is inconsistent with the recognition result of the adjacent activity area video is screened out, and the activity area video collected from the second acquisition time of each target activity area video (the acquisition time of the adjacent activity area video) to the first acquisition time of the target activity area video is used as the traceability video of the captive.

[0099] It should be noted that any one of steps S602 to S622 or any combination of multiple steps can be combined with any one of steps S202 to S208 to form a new implementation method according to the needs of implementation deployment; in addition, according to the needs of actual deployment, any one or multiple technical features can be selected from steps S602 to S622 and combined with any one or multiple technical features provided by steps S202 to S208 to form a new implementation method; or, any one or multiple technical features in steps S602 to S622 can be replaced by any one or multiple technical features provided by steps S202 to S208 to form a new implementation method according to the needs of actual deployment, which will not be elaborated here.

[0100] The following takes the application of a video processing method provided by this embodiment in the scene of generating a tracing video of a domestic animal as an example, combined with Figure 7 The video processing method provided in this embodiment for the scene of generating a tracing video of a domestic animal is further described. Figure 7 The video processing method applied to the scene of tracing video generation of domesticated animals specifically includes the following steps.

[0101] Step S702: obtaining a video of the activity area captured by a camera configured in the activity area of ​​the animal.

[0102] Step S704, determining a corresponding video acquisition strategy according to the acquisition duration of the activity area video, and reading a prompt text of the video acquisition strategy.

[0103] Step S706: the video encoder performs video encoding on the video frame sequence corresponding to the active area video to obtain video features.

[0104] Step S708: the text encoder performs text encoding on the prompt text to obtain text features.

[0105] The execution order of step S706 and step S708 is not limited in this embodiment.

[0106] Step S710, input the video features and text features into the model to perform biological behavior recognition and obtain recognition results.

[0107] Step S712: When the recognition result is not empty, the activity area video is used as the tracing video of the captive animal.

[0108] In addition, steps S702 to S712 can also be replaced by, when it is detected that the detection condition of the video acquisition strategy is triggered, obtaining the activity area video of the captive at at least one acquisition time corresponding to the video acquisition strategy; performing video encoding on the video frame sequence corresponding to each activity area video by a video encoder to obtain video features, and performing text encoding on the prompt text of the video acquisition strategy by an information encoder to obtain text features; inputting the text features and the video features of each activity area video into a large language model for biological behavior recognition, obtaining the recognition results of each activity area video, and using the activity area video whose recognition results are not empty as the traceability video of the captive.

[0109] It should be noted that any one of steps S702 to S712 or any combination of multiple steps can be combined with any one of steps S202 to S208 to form a new implementation method according to the needs of implementation deployment; in addition, according to the needs of actual deployment, any one or multiple technical features can be selected from steps S702 to S712 and combined with any one or multiple technical features provided by steps S202 to S208 to form a new implementation method; or, any one or multiple technical features in steps S702 to S712 can be replaced with any one or multiple technical features provided by steps S202 to S208 to form a new implementation method according to the needs of actual deployment, which will not be repeated here.

[0110] An embodiment of a video processing device provided in this specification is as follows: In the above-mentioned embodiment, a video processing method is provided, and correspondingly, a video processing device is also provided, which will be described below with reference to the accompanying drawings.

[0111] Reference Figure 8, which shows a schematic diagram of the structure of a video processing device provided by this embodiment. Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0112] This embodiment provides a video processing device, including: The video acquisition module 802 is used to acquire the activity area video of the object, determine the video acquisition strategy corresponding to the activity area video, and read the prompt information of the video acquisition strategy; The encoding module 804 is configured to use a video encoder to perform video encoding on the active area video to obtain video features, and an information encoder to perform information encoding on the prompt information to obtain information features; A biological behavior recognition module 806 is used to input the video features and the information features into a model to perform biological behavior recognition and obtain a recognition result; In the case where the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, the video generation module 808 is run, and the video generation module 808 is used to generate the tracing video of the object based on the activity area video.

[0113] In one embodiment, when encoding the information, the following steps are performed: Position encoding is performed on the prompt information to obtain position features, and the prompt information is input into an embedding layer for information conversion to obtain initial information features; The initial information feature and the position feature are subjected to feature fusion, and the fused feature obtained by the feature fusion is input into an autoregressive model for autoregressive calculation to obtain the information code.

[0114] In one embodiment, the biological behavior recognition module 806 inputs the video features and the information features into the model to perform biological behavior recognition and obtains the recognition result operation, and then performs the following steps: Reading historical recognition results of the activity area video associated with the activity area video; When the historical recognition result is inconsistent with the recognition result, it is determined that the recognition result satisfies the traceability condition.

[0115] In one embodiment, the video generation module 808 performs the following steps when generating the source tracing video of the object based on the activity area video: Reading a first acquisition time of the activity area video and a second acquisition time of the associated activity area video, and determining a tracing time interval based on the first acquisition time and the second acquisition time; The activity area video collected within the tracing time interval is determined as the tracing video.

[0116] In one embodiment, when the biometric behavior recognition module 806 determines that the recognition result meets the traceability condition, the biometric behavior recognition module 806 performs the following steps: Reading a time index of a first acquisition time of the activity area video, and detecting whether the time index is a preset index; If so, determining that the identification result satisfies the traceability condition; If not, splitting the associated time interval to obtain at least one sub-time, wherein the associated time interval is determined based on the first acquisition time and the second acquisition time; A tracing video of the object is generated according to the recognition results corresponding to the activity area videos at each sub-time.

[0117] In one embodiment, after inputting the video features and the information features into the model for biometric behavior recognition and obtaining the recognition result, the biometric behavior recognition module 806 further performs the following steps: If the identification result is not empty, determining that the identification result satisfies the traceability condition; Accordingly, when the video generation module 808 generates the source tracing video of the object based on the activity area video, it performs the following steps: The activity area video is determined as the source tracing video.

[0118] In one embodiment, the encoding module 804 further performs the following steps before the video encoder encodes the active area video to obtain the video features: Extracting video frames from the active area video to obtain a video frame sequence; The sequence of video frames is input into the video encoder.

[0119] In the video processing device provided in the present embodiment, during the video processing process, after acquiring the activity area video of the object, the video acquisition strategy corresponding to the activity area video is first determined, and then the prompt information of the video acquisition strategy is read, so that the activity area video is corresponded to the video acquisition strategy, and the prompt information of the video acquisition strategy is used to perform subsequent biological behavior recognition, thereby improving the flexibility and effectiveness of biological behavior recognition on the activity area video. On this basis, the video encoder encodes the activity area video to obtain video features, and the information encoder encodes the prompt information to obtain information features, and then the video features and the information features are input into the model for biological behavior recognition to obtain Recognition results, before using the model to perform biological behavior recognition, encode the activity area video and prompt information respectively to obtain accurate encoding results for the activity area video and prompt information, and then use the encoded video features and information features as input to enable the model to generate behavior recognition, thereby improving the accuracy of the model's biological behavior recognition. With the help of the model, the biological behavior corresponding to the video acquisition strategy is recognized on the activity area video corresponding to the video acquisition strategy. After obtaining the recognition results, the tracing conditions corresponding to the video acquisition strategy are used to affect the generation of the tracing video. In this way, biological behavior recognition and tracing video generation are performed based on the video acquisition strategy, thereby improving the richness and effectiveness of the generated tracing video.

[0120] An embodiment of a video processing device provided in this specification is as follows: Corresponding to the video processing method described above, based on the same technical concept, an embodiment of the present application further provides a video processing device, which is used to execute the video processing method provided above. Fig. 9 A schematic diagram of the structure of a video processing device provided in an embodiment of the present application.

[0121] This embodiment provides a video processing device, including: like Fig. 9As shown, the video processing device may have relatively large differences due to different configurations or performances, and may include one or more processors 901 and memory 902, and the memory 902 may store one or more storage applications or data. Among them, the memory 902 may be a short-term storage or a permanent storage. The application stored in the memory 902 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the video processing device. Furthermore, the processor 901 may be configured to communicate with the memory 902, and execute a series of computer executable instructions in the memory 902 on the video processing device. The video processing device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, one or more keyboards 906, etc.

[0122] In a specific embodiment, the video processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the video processing device, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following: Acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; The video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features; Inputting the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result; When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

[0123] In one embodiment, the processor performs the following steps when encoding the information: Position encoding is performed on the prompt information to obtain position features, and the prompt information is input into an embedding layer for information conversion to obtain initial information features; The initial information feature and the position feature are subjected to feature fusion, and the fused feature obtained by the feature fusion is input into an autoregressive model for autoregressive calculation to obtain the information code.

[0124] In one embodiment, after inputting the video features and the information features into a model for biological behavior recognition and obtaining a recognition result, the processor further performs the following steps: Reading historical recognition results of the activity area video associated with the activity area video; When the historical recognition result is inconsistent with the recognition result, it is determined that the recognition result satisfies the traceability condition.

[0125] In one embodiment, when the processor generates the source tracing video of the object from the activity area video, the processor performs the following steps: Reading a first acquisition time of the activity area video and a second acquisition time of the associated activity area video, and determining a tracing time interval based on the first acquisition time and the second acquisition time; The activity area video collected within the tracing time interval is determined as the tracing video.

[0126] In one embodiment, when the processor determines that the recognition result satisfies the traceability condition, the processor performs the following steps: Reading a time index of a first acquisition time of the activity area video, and detecting whether the time index is a preset index; If so, determining that the identification result satisfies the traceability condition; If not, splitting the associated time interval to obtain at least one sub-time, wherein the associated time interval is determined based on the first acquisition time and the second acquisition time; A tracing video of the object is generated according to the recognition results corresponding to the activity area videos at each sub-time.

[0127] In one embodiment, after inputting the video features and the information features into a model for biological behavior recognition and obtaining a recognition result, the processor further performs the following steps: If the identification result is not empty, determining that the identification result satisfies the traceability condition; Accordingly, in one embodiment, when the processor generates the source tracing video of the object based on the activity area video, the processor performs the following steps: The activity area video is determined as the source tracing video.

[0128] In one embodiment, the processor further performs the following steps before the video encoder performs video encoding on the active area video to obtain the video features: Extracting video frames from the active area video to obtain a video frame sequence; The sequence of video frames is input into the video encoder.

[0129] In the video processing device provided in the present embodiment, during the video processing process, after acquiring the active area video of the object, the video acquisition strategy corresponding to the active area video is first determined, and then the prompt information of the video acquisition strategy is read, so that the active area video is corresponded to the video acquisition strategy, and the subsequent biological behavior recognition is performed through the prompt information of the video acquisition strategy, thereby improving the flexibility and effectiveness of the biological behavior recognition performed on the active area video. On this basis, the video encoder encodes the active area video to obtain video features, and the information encoder encodes the prompt information to obtain information features, and then the video features and the information features are input into the model for biological behavior recognition to obtain Recognition results, before using the model to perform biological behavior recognition, encode the activity area video and prompt information respectively to obtain accurate encoding results for the activity area video and prompt information, and then use the encoded video features and information features as input to enable the model to generate behavior recognition, thereby improving the accuracy of the model's biological behavior recognition. With the help of the model, the biological behavior corresponding to the video acquisition strategy is recognized on the activity area video corresponding to the video acquisition strategy. After obtaining the recognition results, the tracing conditions corresponding to the video acquisition strategy are used to affect the generation of the tracing video. In this way, biological behavior recognition and tracing video generation are performed based on the video acquisition strategy, thereby improving the richness and effectiveness of the generated tracing video.

[0130] An embodiment of a computer-readable storage medium provided in this specification is as follows: Corresponding to the video processing method described above, based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium.

[0131] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, and the computer-executable instructions implement the following process when executed: Acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; The video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features; Inputting the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result; When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

[0132] In the computer-readable storage medium provided in the present embodiment, during the video processing process, after acquiring the active area video of the object, the video acquisition strategy corresponding to the active area video is first determined, and then the prompt information of the video acquisition strategy is read, so that the active area video is matched with the video acquisition strategy, and the prompt information of the video acquisition strategy is used to perform subsequent biological behavior recognition, thereby improving the flexibility and effectiveness of biological behavior recognition for the active area video. On this basis, the video encoder encodes the active area video to obtain video features, and the information encoder encodes the prompt information to obtain information features, and then the video features and the information features are input into the model for biological behavior recognition. The recognition result is obtained. Therefore, before the biological behavior recognition is performed through the model, the activity area video and the prompt information are encoded respectively to obtain accurate encoding results of the activity area video and the prompt information. The video features and information features obtained by encoding are then used as input to enable the model to generate behavior recognition, thereby improving the accuracy of the model's biological behavior recognition. After the biological behavior recognition corresponding to the video acquisition strategy is performed on the activity area video corresponding to the video acquisition strategy with the help of the model, and the recognition result is obtained, the generation of the tracing video is affected by the tracing conditions corresponding to the video acquisition strategy. Therefore, the biological behavior recognition and tracing video generation are performed based on the video acquisition strategy, thereby improving the richness and effectiveness of the generated tracing video.

[0133] It should be noted that the embodiment of a computer-readable storage medium in this specification and the embodiment of a video processing method in this specification are based on the same inventive concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0134] Another embodiment of the present disclosure further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following process is implemented: Acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; The video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features; Inputting the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result; When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

[0135] In the computer program product provided in this embodiment, during the video processing process, after acquiring the activity area video of the object, the video acquisition strategy corresponding to the activity area video is first determined, and then the prompt information of the video acquisition strategy is read, so that the activity area video is corresponded to the video acquisition strategy, and the subsequent biological behavior recognition is performed through the prompt information of the video acquisition strategy, thereby improving the flexibility and effectiveness of the biological behavior recognition performed on the activity area video. On this basis, the video encoder encodes the activity area video to obtain video features, and the information encoder encodes the prompt information to obtain information features, and then the video features and the information features are input into the model for biological behavior recognition to obtain The recognition result is obtained. Therefore, before the model is used for biological behavior recognition, the activity area video and prompt information are encoded respectively to obtain accurate encoding results of the activity area video and prompt information. The video features and information features obtained by encoding are then used as input to enable the model to generate behavior recognition, thereby improving the accuracy of the model's biological behavior recognition. With the help of the model, the biological behavior corresponding to the video acquisition strategy is recognized on the activity area video corresponding to the video acquisition strategy. After obtaining the recognition result, the tracing condition corresponding to the video acquisition strategy is used to affect the generation of the tracing video. Therefore, the video acquisition strategy is used as the basis for biological behavior recognition and tracing video generation, thereby improving the richness and effectiveness of the generated tracing video.

[0136] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned video processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0137] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] It should be understood by those skilled in the art that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the embodiments of the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable test equipment to produce a machine, so that the instructions executed by the processor of the computer or other programmable test equipment generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable test equipment to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, the instruction device being implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable test device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0143] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0144] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0146] Embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0147] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0148] The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims

1. A video processing method, characterized in that: The method comprises: Acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; The video encoder performs video encoding on the active area video to obtain video features, and the information encoder performs information encoding on the prompt information to obtain information features; Inputting the video features and the information features into a model to perform biological behavior recognition to obtain a recognition result; When the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a tracing video of the object is generated based on the activity area video.

2. The method according to claim 1, characterized in that The information encoding includes: Performing position encoding on the prompt information to obtain position features, and performing initial information encoding on the prompt information to obtain initial information features; The initial information feature and the position feature are subjected to feature fusion, and the fused feature obtained by the feature fusion is input into an autoregressive model for autoregressive calculation to obtain the information code.

3. The method according to claim 1, characterized in that After the operation of inputting the video features and the information features into the model for biological behavior recognition and obtaining the recognition result is performed, the method further includes: Reading historical recognition results of the activity area video associated with the activity area video; When the historical recognition result is inconsistent with the recognition result, it is determined that the recognition result satisfies the traceability condition.

4. The method according to claim 3, characterized in that The generating the source tracing video of the object based on the activity area video includes: Reading a first acquisition time of the activity area video and a second acquisition time of the associated activity area video, and determining a tracing time interval based on the first acquisition time and the second acquisition time; The activity area video collected within the tracing time interval is determined as the tracing video.

5. The method according to claim 3, characterized in that: The determining that the identification result satisfies the traceability condition includes: Reading a time index of a first acquisition time of the activity area video, and detecting whether the time index is a preset index; If so, determining that the identification result satisfies the traceability condition; If not, splitting the associated time interval to obtain at least one sub-time, wherein the associated time interval is determined based on the first acquisition time and the second acquisition time; A tracing video of the object is generated according to the recognition results corresponding to the activity area videos at each sub-time.

6. The method according to claim 1, characterized in that After the operation of inputting the video features and the information features into the model for biological behavior recognition and obtaining the recognition result is performed, the method further includes: If the identification result is not empty, determining that the identification result satisfies the traceability condition; The generating the source tracing video of the object based on the activity area video includes: The activity area video is determined as the source tracing video.

7. The method according to claim 1, characterized in that The video encoder performs video encoding on the active area video to obtain the video feature operation before the operation is performed, further comprising: Extracting video frames from the active area video to obtain a video frame sequence; The sequence of video frames is input into the video encoder.

8. A video processing device, characterized in that: The device comprises: A video acquisition module, used to acquire an activity area video of an object, determine a video acquisition strategy corresponding to the activity area video, and read prompt information of the video acquisition strategy; An encoding module, wherein a video encoder performs video encoding on the active area video to obtain video features, and an information encoder performs information encoding on the prompt information to obtain information features; A biological behavior recognition module, used for inputting the video features and the information features into a model to perform biological behavior recognition and obtain a recognition result; In the case where the recognition result satisfies the tracing condition corresponding to the video acquisition strategy, a video generation module is run, and the video generation module is used to generate a tracing video of the object based on the activity area video.

9. A video processing device, characterized in that: The device comprises: A processor; and a memory configured to store computer executable instructions, which, when executed, cause the processor to perform the video processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the video processing method according to any one of claims 1 to 7 is implemented.

Images