Video image archiving method, system and storage medium

Abstract

This application discloses a video image archiving method, system, and storage medium. The video image archiving method includes: identifying a target object from a regional video image; tracking the state of the target object based on multiple frames in the video image to obtain the target object's motion trajectory; matching corresponding action types and generating an action sequence table for the video action; matching behavior descriptions, outputting behavior application files, and identifying the application scenarios of the video images; and calling corresponding language models to label and classify the regional video images based on the behavior application files. This application can flexibly adapt to various usage environments and needs. Through the built-in multiple language models, it automatically switches according to practical scenarios, greatly improving the applicability and convenience of the device, providing users with powerful natural language processing capabilities, and providing important practical value and broad market prospects.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and specifically to a video image archiving method, system and storage medium. Background Art

[0002] With the advancement of information technology, people can easily access a variety of video image data from different scenarios. Consequently, the demand for managing various video files is increasing. Video images are sequences of continuous static images, providing a more vivid and lively description of objective objects. Video image processing systems are inextricably linked to the development of contemporary science and technology. On the one hand, the continuous improvement of processor computing power, the significant increase in storage space, and the diversification of hardware support methods with the development of computer technology have greatly promoted the development of video image processing systems. On the other hand, the rapid development of semiconductor devices and large-scale integrated circuit technology, with its increased processor power and storage capacity, has also promoted the development of video image processing systems.

[0003] During the process of conceiving and developing this application, the applicant identified at least the following technical problems: Traditional video image processing technology primarily uses PCs for real-time image processing. Because PCs are based on the von Neumann complex instruction set computer (CIPC), they essentially execute instructions sequentially and cannot implement parallel processing, resulting in poor real-time performance. Using dedicated DSP chips for real-time image processing, due to their Harvard architecture, which separates data and program space, allows for certain parallel processing and complex algorithms, but cannot implement more complex logical operations and high-speed processing. Video image data is characterized by large data volumes and computational complexity, making its processing extremely time-consuming. This results in chaotic management and inconvenient retrieval of large numbers of video image files. Existing file processing equipment often relies on internet connections, which not only lacks security but also cannot be used in environments without an internet connection. Furthermore, the efficiency and accuracy of existing equipment in processing video image files needs to be improved. Therefore, there is an urgent need for a secure and efficient method for collecting, processing, storing, and archiving video image files. Summary of the Invention

[0004] In order to alleviate the above technical problems, the purpose of this application is to provide a video image archiving method comprising:

[0005] In response to acquiring a video image of a preset area, identifying a target object from the video image of the area;

[0006] Tracking the state of the target object based on multiple frames of the video image to obtain a motion trajectory of the target object;

[0007] Performing video action analysis on the motion trajectory to match the corresponding action type and generating an action sequence table of the video action;

[0008] Matching behavior descriptions according to the action sequence table, outputting a behavior description file, and identifying application scenarios of the video images;

[0009] Based on the application scenario, the corresponding language model is called to label, classify and archive the regional video images according to the behavior description file.

[0010] Optionally, in response to acquiring a video image of a preset area, the step of identifying a target object from the video image of the area includes:

[0011] Obtain each frame of the video image, and convolve the frame image using a convolution kernel in a convolutional neural network to obtain an intermediate matrix;

[0012] Correcting the deviation of each element of the intermediate matrix to extract feature points of the frame image based on an activation function;

[0013] Based on a preset object image library, the object image library is traversed to screen out at least one image library object that matches the feature point as the target object.

[0014] Optionally, the step of tracking the state of the target object based on multiple frames of images in the video image to obtain the motion trajectory of the target object includes:

[0015] Based on the time sequence of the multiple frames of images, extracting multiple frames of images containing the target object and arranging them as a target processing flow;

[0016] Performing state recognition of a target object on the first frame image based on a preset image recognition model, and obtaining initial position information and initial state information of the target object in the first frame image;

[0017] For any image except the first frame, tracking the intermediate position information of the target object in each frame according to the position information of the target object in the previous frame of the corresponding image;

[0018] According to the intermediate position information of the target object in each frame image and the time of each frame image, a motion trajectory of the target object under the corresponding state information is generated.

[0019] Optionally, the step of performing video action analysis on the motion trajectory to match the corresponding action type and generating an action sequence table of the video action includes:

[0020] Identifying a target part of the target object in each frame of image, and identifying an action posture in the frame of image based on a relative position relationship of the target part;

[0021] According to the motion posture of the multiple frames of images, a preset motion model library is retrieved to determine the motion type of the target object;

[0022] The action postures of the multiple frames of images are sorted based on the image time of each frame of image, and an action sequence table of the video action is established.

[0023] Optionally, the step of matching behavior descriptions according to the action sequence table, outputting a behavior description file, and identifying an application scenario of the video image includes:

[0024] Reading the action sequence table, and generating an action description of the action posture in each frame of image through a pre-trained description neural network based on the positional relationship and time difference between the action posture in the previous frame of image and / or the action posture in the next frame of image;

[0025] Segmenting the action descriptions of the multiple frames of images to obtain multiple action words;

[0026] Extracting the first description sentence corresponding to each action word from a pre-established action description library;

[0027] performing scene cluster analysis based on the plurality of first description sentences corresponding to the plurality of action words, and obtaining a cluster feature scene with the highest probability as the application scene of the video image;

[0028] Corresponding to each action word, a second description sentence that meets the application scenario is extracted from the action description library to generate the behavior description file.

[0029] Optionally, the step of calling a corresponding language model based on the application scenario to label, classify and archive the regional video images according to the behavior description file includes:

[0030] Retrieving a target language model corresponding to the application scenario from a pre-trained language model library;

[0031] Reading the behavior description file, and performing semantic understanding on the behavior description file based on the target language model;

[0032] Based on the semantic understanding, assigning labels to the behavior description file;

[0033] The behavior description files are classified and filed according to the tag assignments.

[0034] Optionally, before the step of retrieving the target language model corresponding to the application scenario from the pre-trained language model library, the method further includes:

[0035] Building a text database, and using a transformer neural network to analyze the conditional probability of each word in the text database corresponding to other words, so as to train the transformer neural network to establish target language models corresponding to different scenarios;

[0036] The trained target language model is exposed to a preset amount of preset task data, so that the target language model is fine-tuned for the preset task data.

[0037] Optionally, the language model is a GPT large language model.

[0038] The present application also provides a video image archiving system, the video image archiving system comprising a processor and a memory;

[0039] The memory stores a computer program, and when the computer program is executed by the processor, the steps of the video image archiving method as described above are implemented.

[0040] Optionally, the video image archiving system includes:

[0041] An object recognition module, for identifying target objects from a regional video image;

[0042] A state tracking module is configured to track the state of the target object based on multiple frames of the video image to obtain a motion trajectory of the target object;

[0043] An action analysis module is used to perform video action analysis on the motion trajectory to match the corresponding action type and generate an action sequence table of the video action;

[0044] A file generation module is used to match behavior descriptions according to the action sequence table, output a behavior description file, and identify the application scenario of the video image;

[0045] The classification and archiving module is used to call the corresponding language model based on the application scenario to label, classify and archive the regional video images according to the behavior description file.

[0046] The present application also provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the video image archiving method as described above are implemented.

[0047] Compared with the prior art, the present application identifies the target object from the video image of the preset area in response to obtaining the video image of the area; tracks the state of the target object based on multiple frames in the video image to obtain the motion trajectory of the target object; performs video action analysis on the motion trajectory to match the corresponding action type and generate an action sequence table for the video action; matches the behavior description according to the action sequence table, outputs a behavior description file, and identifies the application scenario of the video image; based on the application scenario, calls the corresponding language model to label, classify and archive the video image of the area according to the behavior description file. This application can flexibly adapt to various usage environments and needs through an artificial intelligence computing device that is compatible with multiple large language models and feature libraries. It automatically switches according to practical scenarios through built-in multiple language models, greatly improving the applicability and convenience of the device; it has sufficient built-in computing power to provide users with powerful natural language processing capabilities, providing important practical value and broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without inventive work.

[0049] Figure 1 This is a flow chart of a video image archiving method according to an embodiment of the present application.

[0050] The purpose of this application, its features, and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and the accompanying text are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of this application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0053] It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if," as used herein, may be interpreted as "upon," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the recited features, steps, operations, elements, components, items, types, and / or groups, but do not preclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or," "and / or," "including at least one of the following," etc., as used herein, may be interpreted as inclusive, meaning any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”; and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0054] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0055] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0056] It should be noted that in this article, step codes such as S10 and S20 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S20 first and then S10, etc., but these should all be within the scope of protection of this application.

[0057] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0058] This application first provides a video image archiving method, Figure 1 This is a flow chart of a video image archiving method according to an embodiment of the present application.

[0059] like Figure 1 As shown, the video image archiving method includes:

[0060] S10: In response to acquiring a video image of a preset area, identifying a target object from the video image of the area.

[0061] For example, a video image is a sequence of continuous static images, providing a more vivid and vivid description of objective objects. Video images can be captured by various electronic devices equipped with photosensitive devices such as cameras and still cameras. The video images captured and stored by electronic devices often contain continuous image frames of specific objects. The target object is at least one specific object that can be used to analyze the content of the video image.

[0062] S20: Tracking the state of the target object based on multiple frames of the video image to obtain a motion trajectory of the target object.

[0063] For example, the digitized video image is composed of multiple frames arranged in a time sequence. The motion trajectory of a specific target object in the time sequence can be analyzed by using the multiple frames connected together in time.

[0064] S30: Performing video action analysis on the motion trajectory to match the corresponding action type, and generating an action sequence table of the video action.

[0065] For example, by analyzing the motion trajectory of the target object, the action type of the target object in this segment of image frames can be determined, and then the action sequence table in the video image can be determined.

[0066] S40: matching behavior descriptions according to the action sequence table, outputting a behavior description file, and identifying application scenarios of the video image.

[0067] For example, by analyzing the target object's actions at different times, the target object's actions during this period can be determined, thereby forming a textual description and identifying the current scene of the target object's environment.

[0068] S50: Based on the application scenario, a corresponding language model is called to label, classify and archive the regional video images according to the behavior description file.

[0069] For example, based on the application scenario reflected in the video image and the description of the target object's behavior, the language model can label and classify the video image, thereby facilitating data archiving and subsequent query use.

[0070] In this embodiment, by being compatible with multiple large language models and feature libraries, it can flexibly adapt to various usage environments and needs. Through the built-in multiple language models, it automatically switches according to practical scenarios, thereby improving the applicability and convenience of the device, providing users with powerful natural language processing capabilities, and providing important practical value and broad market prospects.

[0071] Optionally, in response to acquiring a video image of a preset area, the step of identifying a target object from the video image of the area includes:

[0072] Obtain each frame of the video image, and convolve the frame image using a convolution kernel in a convolutional neural network to obtain an intermediate matrix;

[0073] Correcting the deviation of each element of the intermediate matrix to extract feature points of the frame image based on an activation function;

[0074] Based on a preset object image library, the object image library is traversed to screen out at least one image library object that matches the feature point as the target object.

[0075] For example, a Sobel convolution kernel can be used to convolve each image frame, generating an intermediate matrix. A bias value is then added to each element of the intermediate matrix, corrected for bias, and then fed into an activation function to generate feature points at different convolutional layers, ultimately identifying the feature points of the image frame.

[0076] For example, based on the feature points of different convolutional layers, the feature points of the target object in the frame image can be determined. This is done by searching the object database, retrieving the feature points of matching image library objects, and sequentially comparing the similarity scores between the object to be identified and each image in the image library. The N image classes with the closest similarity scores are then screened out. The image of the object to be identified and the image to be screened are then fed into a convolutional neural network, where feature vectors are extracted through the convolutional and fully connected layers. Principal component analysis is then used to combine the extracted linearly correlated feature vectors to reduce the linear correlation between the feature vectors and reduce feature redundancy. The feature vectors of the feature combinations combined through principal component analysis are hashed, and finally the Hamming distance between the hash codes of the object to be identified and the image to be screened is calculated. The smaller the distance, the higher the image similarity, thereby determining the target object in each frame. In this way, the object recognition accuracy of each frame image can be improved, making it easier to find the target object more accurately.

[0077] Optionally, the step of tracking the state of the target object based on multiple frames of images in the video image to obtain the motion trajectory of the target object includes:

[0078] Based on the time sequence of the multiple frames of images, extracting multiple frames of images containing the target object and arranging them as a target processing flow;

[0079] Performing state recognition of a target object on the first frame image based on a preset image recognition model, and obtaining initial position information and initial state information of the target object in the first frame image;

[0080] For any image except the first frame, tracking the intermediate position information of the target object in each frame according to the position information of the target object in the previous frame of the corresponding image;

[0081] According to the intermediate position information of the target object in each frame image and the time of each frame image, a motion trajectory of the target object under the corresponding state information is generated.

[0082] Exemplarily, according to the shooting time sequence of each frame image, multiple frames of images are sequentially treated as a processing batch, and the first frame image is first subjected to object recognition and state recognition processing to obtain the position information and state information of each target object included in the first frame image. Then, for other frame images except the first frame image, the position information of each target object in the frame image is tracked based on the position information of each target object in the previous frame image of the frame image. Based on the position information of each target object in each frame image, the motion trajectory of each target object under the corresponding state information is generated. This embodiment can identify the target objects existing in the image while determining the state of each target object and tracking the motion trajectory of each target object, thereby realizing continuous state tracking of each target object in the video image.

[0083] Optionally, the step of performing video action analysis on the motion trajectory to match the corresponding action type and generating an action sequence table of the video action includes:

[0084] Identifying a target part of the target object in each frame of image, and identifying an action posture in the frame of image based on a relative position relationship of the target part;

[0085] According to the motion posture of the multiple frames of images, a preset motion model library is retrieved to determine the motion type of the target object;

[0086] The action postures of the multiple frames of images are sorted based on the image time of each frame of image, and an action sequence table of the video action is established.

[0087] Exemplarily, the action of the target object in the current captured image is identified, including: identifying the target part of the target object in the current captured image; identifying the action of the target part to obtain a corresponding action category; if the action category of the target part is consistent with the action category of the wake-up action corresponding to the target function, then it is determined that the action of the target object matches the wake-up action corresponding to the target function.

[0088] The target object's movements can be those performed by any body part, such as the eyes, mouth, hands, head, torso, or feet, or any combination of two or more body parts. Specifically, during the process of identifying the target part, the body part's movement amplitude can be identified. Based on the results of the movement amplitude identification, the body part with a movement amplitude change greater than a preset threshold is selected as the target part.

[0089] The process of identifying the target body's movements specifically involves: first, modeling the human body's joints. This involves treating the various limbs as an internally connected, rigid system consisting of bones and joints. The relative motion of the bones and joints constitutes changes in human posture. Within the body's numerous joints, the backbone is abstracted into joints based on differences in the amplitude of movement. For example, the spine is abstracted into three joints: the neck, chest, and abdomen. This results in a human model where the upper body includes the head, neck, chest, abdomen, two upper arms, and two lower arms, while the lower body includes two thighs and two lower legs. For each of the multiple selected movement types, the normal human body expression of each common movement is selected, and the limb responses are analyzed in detail. Since the human body is abstracted into a rigid body model, the first thing is the movement of the human body's center of gravity, which is divided into forward, backward and natural states; in addition to the movement of the center of gravity, the second is the rotation of the joints, and the human body undergoes movement changes. The relevant joints include the head, chest, shoulders and elbows. The corresponding movements are the bending of the head, the rotation of the chest, the swing and extension direction of the upper arm, and the bending of the elbow. These parameters, combined with the movement of the upper center of gravity, include a total of seven degrees of freedom of movement, which can express the movement of a person's upper body. The movement methods of other limbs can be deconstructed in a similar way. In this way, the type of movement can be identified based on the content of the movement of the user's limbs.

[0090] For example, multiple action postures in multiple frames of images are distributed at different moments. By sorting the multiple different action postures in each frame of images in chronological order, an action sequence table of the video action can be obtained for subsequent analysis of the overall action of the target object.

[0091] Optionally, the step of matching behavior descriptions according to the action sequence table, outputting a behavior description file, and identifying an application scenario of the video image includes:

[0092] Reading the action sequence table, and generating an action description of the action posture in each frame of image through a pre-trained description neural network based on the positional relationship and time difference between the action posture in the previous frame of image and / or the action posture in the next frame of image;

[0093] Segmenting the action descriptions of the multiple frames of images to obtain multiple action words;

[0094] Extracting the first description sentence corresponding to each action word from a pre-established action description library;

[0095] performing scene cluster analysis based on the plurality of first description sentences corresponding to the plurality of action words, and obtaining a cluster feature scene with the highest probability as the application scene of the video image;

[0096] Corresponding to each action word, a second description sentence that meets the application scenario is extracted from the action description library to generate the behavior description file.

[0097] Optionally, the description neural network includes a first memory network, a second memory network, a first fully connected layer, a third memory network, a second fully connected layer and a logistic regression model connected in sequence. Optionally, the description neural network includes a description generation module and a reinforcement learning module. The description generation module is used to generate a description language for the target image, and the reinforcement learning module is used to optimize the loss value so that the description generation module of the description neural network can continuously iterate to select the model parameters with the best effect. Exemplarily, the action description of the action posture in each frame of the image is generated by calculating the time difference between the positional relationships in the action sequence table by the description neural network.

[0098] For example, the action description text can be first segmented to obtain multiple words, and then the first description sentence of each action word is extracted from the action description library. The first description sentences are obtained by matching each word separately, so there may not be much correlation between them. Therefore, the multiple first description sentences are clustered according to various possible scenarios to determine the application scenario with the highest commonality among the multiple first description sentences. At this time, based on the determined application scenario, the second description sentence of each action word is extracted from the action description library again for each word. At this time, all the second description sentences are based on the determined application scenario, so there is considerable correlation between them to express an appropriate description of the video action in the video image.

[0099] Optionally, the step of calling a corresponding language model based on the application scenario to label, classify and archive the regional video images according to the behavior description file includes:

[0100] Retrieving a target language model corresponding to the application scenario from a pre-trained language model library;

[0101] Reading the behavior description file, and performing semantic understanding on the behavior description file based on the target language model;

[0102] Based on the semantic understanding, assigning labels to the behavior description file;

[0103] The behavior description files are classified and filed according to the tag assignments.

[0104] For example, the practice of labeling exists in the metadata system of a distributed file system and is a tool for business practice. An application scenario refers to the environment in which a user is most likely to be when an application is used, including aspects such as time, space, device support, background conditions, social roles, and user emotions. Optionally, a file can be given an original label in the metadata system, such as "dance studio", "mobile phone repair shop", or "dental clinic". During processing and parsing, a process label can be added, such as "Waltz Act III", "Remove the back cover", or "Manual mouthwash required". After processing, a result label can be added, such as "Class dismissed", "Repeat customer", or "Large dental project". In actual applications, there may not be a model trained for every specific scenario, but the model can be trained and adjusted to meet the needs of as many specific scenarios as possible.

[0105] For example, after the application scenario is determined, the target language model that is more suitable for the application scenario can be called in a targeted manner. By building in multiple large language models, it is possible to automatically switch according to different usage scenarios, provide personalized services, flexible applications, and strong adaptability. Optionally, the language model is a GPT large language model. For example, in addition to using large language models such as GPT-3, some open source, small language models can also be considered. These models may perform better in certain specific tasks or certain targeted application scenarios, or provide higher performance in resource-limited environments. Optionally, although automatic switching of language models provides convenience for users, in some cases, users may want to manually select and switch language models to obtain a higher degree of control and personalization.

[0106] For example, after reading an action description file, when given a prompt or input text, GPT uses cases and its transformer network to analyze the context of each word or phrase and generate its representation. This helps the model understand the meaning and relationships between words in the input text. Based on its contextual analysis of the input text, it generates human-like text in response to the prompt. The language model uses its understanding of language tasks and the relationships between words and phrases to predict the most likely next word or phrase, thereby providing a semantic understanding of the action description file.

[0107] Exemplarily, based on the understanding of content at the semantic level, file classification, labeling and archiving can be performed during the file output process. However, in some embodiments, it can be considered to expand the output module to provide more diverse output forms, such as automatic translation of files, automatic summaries of files, etc. In the storage process after classification, it is used to store directly obtained files and directly received files. In the future, with the development of storage technology and the reduction of costs, more advanced storage technologies can be considered, such as cloud-based storage, distributed storage, etc. Using large language model processing, various types of files can be accurately identified and processed, greatly improving the accuracy and efficiency of processing. Through the classification, labeling and archiving functions of file output, files such as video images are easy to retrieve and use, which improves work efficiency. In addition, the independent operation mode of the device also increases the safety of use.

[0108] Optionally, before the step of retrieving the target language model corresponding to the application scenario from the pre-trained language model library, the method further includes:

[0109] Building a text database, and using a transformer neural network to analyze the conditional probability of each word in the text database corresponding to other words, so as to train the transformer neural network to establish target language models corresponding to different scenarios;

[0110] The trained target language model is exposed to a preset amount of preset task data, so that the target language model is fine-tuned for the preset task data.

[0111] For example, in the process of training the target language model, a large amount of text data from the internet, including books, articles, and websites, can be pre-trained. In this process, the model uses a transformer neural network to analyze the context of each word or phrase and generate a representation that predicts the next word in the sentence. For example, GPT calculates how likely a certain word is to appear in a text given another word in this text. It is called the conditional probability of the word. Once pre-training is completed, it can be fine-tuned for a specific task by exposing it to a smaller amount of task-specific data. This fine-tuning process helps the model learn how to perform a specific task, such as language translation or code generation like Python, by adjusting its algorithm to better adapt to new data.

[0112] The present application also provides a video image archiving system, the video image archiving system comprising a processor and a memory;

[0113] The memory stores a computer program, and when the computer program is executed by the processor, the steps of the video image archiving method as described above are implemented.

[0114] For example, when building an AI computing power archiving system based on a large language model and feature library, a distributed file system and a supporting metadata system can be first established as a database. Various files and documents are automatically or manually collected into the system. The system infers the true file format based on the input file's extension, metadata, attribute information, and binary header, such as text files (txt), comma-delimited files (csv), hypertext files (html), images, audio files, video files, portable document formats (pdf), and Office documents. Based on the inference results, a format parser corresponding to each file is selected and parsed. For example, text files can be directly extracted without additional parsing. Other text-like formats can be processed directly as text files or using their corresponding parsing tools. Other binary documents such as pdf, docx, and xlsx require parsing using official or third-party APIs. Image files require pattern recognition and optical character recognition (OCR). Audio and video files attempt to extract and recognize the audio track. In short, the goal of the second step is to convert the input file into a text file as much as possible. Different parsing tools, application models, and focus on the same file can lead to different parsing results. After conversion is complete, the source file, inference results, adapted parser, parsing results, and version can be saved and archived to a distributed file system and supporting metadata system. Based on the needs of the business scenario, relevant documents within the scenario are selected, packaged, and pushed to the input interface of a large language model such as GPT for analysis, processing, and iteration.

[0115] Optionally, the video image archiving system includes:

[0116] An object recognition module, for identifying target objects from a regional video image;

[0117] A state tracking module is configured to track the state of the target object based on multiple frames of the video image to obtain a motion trajectory of the target object;

[0118] An action analysis module is used to perform video action analysis on the motion trajectory to match the corresponding action type and generate an action sequence table of the video action;

[0119] A file generation module is used to match behavior descriptions according to the action sequence table, output a behavior description file, and identify the application scenario of the video image;

[0120] The classification and archiving module is used to call the corresponding language model based on the application scenario to label, classify and archive the regional video images according to the behavior description file.

[0121] In one embodiment, the AI ​​computing power archiving system based on a large language model and feature library can also receive both paper and electronic documents. Paper documents are scanned and converted to electronic form using a built-in high-precision scanner. The storage module is used to store both directly obtained and directly received electronic documents. The language model processing module can parse, identify, and process the documents based on the large language model. Finally, the file output module can classify, label, and archive the documents based on the processing results for subsequent retrieval and use.

[0122] The system has the following innovations and advantages:

[0123] Independent operation: The device can operate without an Internet connection, which increases the usage scenarios and safety;

[0124] Automatic processing: The device can automatically receive, scan, store, process and archive documents, reducing manual operations and improving efficiency;

[0125] Efficient and accurate: The processing module based on a large language model can accurately identify and process various types of files, improving processing accuracy;

[0126] Easy to retrieve: The classification, labeling and archiving functions of the file output module make files easy to retrieve and use.

[0127] By using the system of this embodiment, paper and electronic documents can be efficiently collected, processed, stored, and archived. Using a large language model processing module, various types of files can be accurately identified and processed, significantly improving processing accuracy and efficiency. The classification, labeling, and archiving functions of the file output module make files easier to retrieve and use, improving work efficiency. Furthermore, the device's independent operation increases user safety.

[0128] The present application also provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the video image archiving method as described above are implemented.

[0129] Compared with the prior art, the present application identifies the target object from the video image of the preset area in response to obtaining the video image of the area; tracks the state of the target object based on multiple frames in the video image to obtain the motion trajectory of the target object; performs video action analysis on the motion trajectory to match the corresponding action type and generate an action sequence table for the video action; matches the behavior description according to the action sequence table, outputs a behavior description file, and identifies the application scenario of the video image; based on the application scenario, calls the corresponding language model to label, classify and archive the video image of the area according to the behavior description file. This application can flexibly adapt to various usage environments and needs through an artificial intelligence computing device that is compatible with multiple large language models and feature libraries. It automatically switches according to practical scenarios through built-in multiple language models, greatly improving the applicability and convenience of the device; it has sufficient built-in computing power to provide users with powerful natural language processing capabilities, providing important practical value and broad market prospects.

[0130] In the embodiments of the system and storage medium provided in this application, all technical features of any of the above-mentioned terminal control method embodiments may be included. The expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods and will not be repeated here.

[0131] An embodiment of the present application further provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer executes the methods in the various possible implementation modes described above.

[0132] An embodiment of the present application also provides a chip, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that a device equipped with the chip executes the methods in the various possible implementation modes as described above.

[0133] It is understood that the above scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0134] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0135] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0136] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0137] In this application, the same or similar terminology, technical solutions and / or application scenario descriptions are generally only described in detail the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of this application, for the same or similar terminology, technical solutions and / or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

[0138] In this application, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0139] The various technical features of the technical solution of this application can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0140] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0141] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, storage disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0142] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A video image archiving method, characterized in that: include: In response to acquiring a video image of a preset area, identifying a target object from the video image of the area; Tracking the state of the target object based on multiple frames of the video image to obtain a motion trajectory of the target object; Performing video action analysis on the motion trajectory to match the corresponding action type and generating an action sequence table of the video action; Matching behavior descriptions according to the action sequence table, outputting a behavior description file, and identifying application scenarios of the video images; Based on the application scenario, calling a corresponding language model to label and classify the regional video image according to the behavior description file; performing video action analysis on the motion trajectory to match the corresponding action type and generating an action sequence table for the video action includes: Identifying a target part of the target object in each frame of image, and identifying an action posture in the frame of image based on a relative position relationship of the target part; According to the motion posture of the multiple frames of images, a preset motion model library is retrieved to determine the motion type of the target object; The steps of sorting the action postures of multiple frames of images based on the image moment of each frame of image and establishing an action sequence table of the video action; matching the action descriptions according to the action sequence table, outputting the action description file, and identifying the application scenario of the video image include: Reading the action sequence table, and generating an action description of the action posture in each frame of image through a pre-trained description neural network based on the positional relationship and time difference between the action posture in the previous frame of image and / or the action posture in the next frame of image; Segmenting the action descriptions of the multiple frames of images to obtain multiple action words; Extracting the first description sentence corresponding to each action word from a pre-established action description library; performing scene cluster analysis based on the plurality of first description sentences corresponding to the plurality of action words, and obtaining a cluster feature scene with the highest probability as the application scene of the video image; Corresponding to each action word, a second description sentence that meets the application scenario is extracted from the action description library to generate the behavior description file.

2. A video image archiving method according to claim 1, characterized in that: In response to acquiring a video image of a preset area, the step of identifying a target object from the video image of the area includes: Obtain each frame of the video image, and convolve the frame image using a convolution kernel in a convolutional neural network to obtain an intermediate matrix; Correcting the deviation of each element of the intermediate matrix to extract feature points of the frame image based on an activation function; Based on a preset object image library, the object image library is traversed to select at least one image library object that matches the feature points as the target object.

3. A video image archiving method according to claim 2, characterized in that: The step of tracking the state of the target object based on multiple frames of the video image to obtain the motion trajectory of the target object includes: Based on the time sequence of the multiple frames of images, extracting multiple frames of images containing the target object and arranging them as a target processing flow; Performing state recognition of a target object on the first frame image based on a preset image recognition model, and obtaining initial position information and initial state information of the target object in the first frame image; For any image except the first frame, tracking the intermediate position information of the target object in each frame according to the position information of the target object in the previous frame of the corresponding image; According to the intermediate position information of the target object in each frame image and the time of each frame image, a motion trajectory of the target object under the corresponding state information is generated.

4. A video image archiving method according to claim 3, characterized in that: The step of calling the corresponding language model based on the application scenario to label, classify and archive the regional video images according to the behavior description file includes: Retrieving a target language model corresponding to the application scenario from a pre-trained language model library; Reading the behavior description file, and performing semantic understanding on the behavior description file based on the target language model; Based on the semantic understanding, assigning labels to the behavior description file; The behavior description files are classified and filed according to the tag assignments.

5. A video image archiving method according to claim 4, characterized in that: Before the step of retrieving the target language model corresponding to the application scenario from the pre-trained language model library, the following steps are included: Building a text database, and using a transformer neural network to analyze the conditional probability of each word in the text database corresponding to other words, so as to train the transformer neural network to establish target language models corresponding to different scenarios; The trained target language model is exposed to a preset amount of preset task data, so that the target language model is fine-tuned for the preset task data.

6. A video image archiving method according to claim 5, characterized in that: The language model is the GPT large language model.

7. A video image archiving system, characterized in that: The video image archiving system includes a processor and a memory; The memory stores a computer program, and when the computer program is executed by the processor, the steps of the video image archiving method according to any one of claims 1 to 6 are implemented.

8. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the video image archiving method according to any one of claims 1 to 6.

Images