Scene understanding-based video processing method and apparatus, and intelligent analysis system

Abstract

A scene understanding-based video processing method and apparatus, and an intelligent analysis system. In the video processing method, first, an image to be detected of a capturing location is acquired; then, text description information of said image is acquired by using a pre-trained image-text conversion model, wherein the text description information is a text description on said image in terms of image quality, image content, and image elements; the text description information is then inputted into a pre-trained scene understanding model, wherein the scene understanding model is capable of outputting algorithm adaptation information corresponding to the capturing location; and accordingly, a target intelligent algorithm can be loaded, on the basis of the obtained algorithm adaptation information, for a video to be detected captured by a capturing device (for example, a camera) arranged at the capturing location. The target intelligent algorithm can be automatically loaded, and there is no need for manual intelligent algorithm configuration, thereby improving the arrangement efficiency of the capturing device, the configuration efficiency, and the intelligent analysis quality.

Description

Scene-understanding-based video processing methods, devices, and intelligent analysis systems

[0001] This application claims priority to Chinese Patent Application No. 202410924492.7, filed on July 11, 2024, entitled "Video Processing Method, Apparatus and Intelligent Analysis System Based on Scene Understanding", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a video processing method, apparatus and intelligent analysis system based on scene understanding. Background Technology

[0003] Currently, in many scenarios, it is necessary to use cameras or other shooting devices to capture the scene.

[0004] In the relevant solutions, after placing cameras and other subjects at the shooting locations, it is necessary to manually select a suitable intelligent algorithm for that location and manually adjust the parameters of the intelligent algorithm. The intelligent algorithm is used to intelligently analyze the video captured when the subject is at the shooting location.

[0005] However, the above-mentioned method of manually configuring intelligent algorithms can affect the effectiveness of the algorithms if the personnel configuring them are not highly skilled. It is also labor-intensive and prone to errors. This is especially true for large projects that may be equipped with thousands or even tens of thousands of cameras, where manual configuration of intelligent algorithms is less efficient. Summary of the Invention

[0006] To address at least one aspect of the aforementioned technical problems, embodiments of this application provide a video processing method, apparatus, and intelligent analysis system based on scene understanding. The video processing method first acquires textual description information of the image to be detected at the shooting location. Then, based on the textual description information, it obtains algorithm adaptation information for the corresponding shooting location through a scene understanding model. Thus, based on the algorithm adaptation information, the target intelligent algorithm is loaded onto the video to be detected captured by the shooting subject at the shooting location, eliminating the need for manual configuration of the intelligent algorithm and improving the efficiency of shooting subject deployment, configuration, and intelligent analysis quality.

[0007] In a first aspect, embodiments of this application provide a video processing method based on scene understanding, the video processing method comprising:

[0008] The image to be detected is obtained at the shooting point, wherein the image to be detected is captured by the shooting subject deployed at the shooting point;

[0009] A pre-trained image-to-text conversion model is used to obtain textual description information of the image to be detected; wherein, the textual description information describes the image to be detected in terms of image quality, image content, and image elements.

[0010] The text description information is input into a pre-trained scene understanding model, and the algorithm adaptation information corresponding to the shooting location is obtained from the scene understanding model; wherein, the algorithm adaptation information includes at least the algorithm type;

[0011] Based on the algorithm adaptation information, the target intelligent algorithm is loaded onto the video to be detected captured by the subject at the shooting location.

[0012] In one embodiment, the image-to-text conversion model includes a pre-trained image quality assessment sub-model, an image content extraction sub-model, and an image element parsing sub-model;

[0013] The step of obtaining the text description information of the image to be detected using a pre-trained image-to-text conversion model includes:

[0014] The image to be detected is input into the image quality assessment sub-model, the image content extraction sub-model, and the image element parsing sub-model, respectively, and the first text information output by the image quality assessment sub-model, the second text information output by the image content extraction sub-model, and the third text information output by the image element parsing sub-model are obtained respectively; wherein, the first text information reflects the textual description of the image quality, the second text information reflects the textual description of the image content, and the third text information reflects the textual description of the image elements.

[0015] In one embodiment, the first text information describes the image quality of the image to be detected from multiple dimensions, including at least sharpness, brightness, chroma, and occlusion.

[0016] The second text information describes the objects in the image to be detected, their attributes, and the positional relationships between them.

[0017] The third text information describes the objects in the image to be detected in terms of their categories, the location of each category of objects in the image, and the proportion of each category of objects in the image to be detected.

[0018] In one embodiment, the step of inputting the text description information into a pre-trained scene understanding model and obtaining the algorithm adaptation information corresponding to the shooting location output by the scene understanding model includes:

[0019] The first text information, the second text information, and the third text information are simultaneously input into the scene understanding model;

[0020] Obtain the algorithm adaptation information output by the scene understanding model, wherein the algorithm adaptation information includes the algorithm type corresponding to the shooting location, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

[0021] In one embodiment, before the step of obtaining the algorithm adaptation information output by the scene understanding model, the method further includes:

[0022] Obtain the quality level of the shooting location output by the scene understanding model based on the first text information;

[0023] If the quality level is lower than the quality level threshold, the maintenance information of the shooting location shall be reported.

[0024] In one embodiment, after the step of loading the target intelligent algorithm onto the video to be detected captured by the subject at the shooting location according to the algorithm adaptation information, the video processing method further includes:

[0025] The intelligent analysis results of the target intelligent algorithm on the video to be detected are obtained; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0026] The fourth text information corresponding to the intelligent analysis result is input into the scene understanding model, and the algorithm rule adjustment parameters corresponding to the algorithm type are output by the scene understanding model.

[0027] The parameters are adjusted according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and the adjusted target intelligent algorithm is loaded onto the video to be detected.

[0028] In one embodiment, after obtaining the intelligent analysis results of the target intelligent algorithm on the video to be detected, the video processing method further includes:

[0029] Based on the comparison between the text description information and the preset information, and the comparison between the statistical information in the intelligent analysis results and the preset information, camera parameter adjustment information is generated, and the parameters of the subject being photographed are adjusted according to the camera parameter adjustment information.

[0030] Secondly, embodiments of this application provide a scene-understanding-based video processing apparatus, the video processing apparatus comprising:

[0031] The video shooting module is used to acquire the image to be detected at the shooting point, wherein the image to be detected is captured by the shooting subject deployed at the shooting point;

[0032] The image-to-text conversion module is used to obtain textual description information of the image to be detected using a pre-trained image-to-text conversion model; wherein, the textual description information describes the image to be detected in terms of image quality, image content, and image elements.

[0033] The scene understanding module is used to input the text description information into a pre-trained scene understanding model and obtain the algorithm adaptation information corresponding to the shooting location output by the scene understanding model; wherein, the algorithm adaptation information includes at least the algorithm type;

[0034] The algorithm loading module is used to load the target intelligent algorithm onto the video to be detected captured by the subject at the shooting location based on the algorithm adaptation information.

[0035] In one embodiment, the image-to-text conversion model includes a pre-trained image quality assessment sub-model, an image content extraction sub-model, and an image element parsing sub-model;

[0036] The image-to-text conversion module is specifically used for:

[0037] The image to be detected is input into the image quality assessment sub-model, the image content extraction sub-model, and the image element parsing sub-model, respectively, and the first text information output by the image quality assessment sub-model, the second text information output by the image content extraction sub-model, and the third text information output by the image element parsing sub-model are obtained respectively; wherein, the first text information reflects the textual description of the image quality, the second text information reflects the textual description of the image content, and the third text information reflects the textual description of the image elements.

[0038] In one embodiment, the first text information describes the image quality of the image to be detected from multiple dimensions, including at least sharpness, brightness, chroma, and occlusion.

[0039] The second text information describes the objects in the image to be detected, their attributes, and the positional relationships between them.

[0040] The third text information describes the objects in the image to be detected in terms of their categories, the location of each category of objects in the image, and the proportion of each category of objects in the image to be detected.

[0041] In one embodiment, the scene understanding module includes:

[0042] The text input submodule is used to simultaneously input the first text information, the second text information, and the third text information into the scene understanding model;

[0043] The algorithm acquisition submodule is used to acquire the algorithm adaptation information output by the scene understanding model, wherein the algorithm adaptation information includes the algorithm type corresponding to the shooting point, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

[0044] In one embodiment, the scene understanding module further includes:

[0045] The quality level acquisition submodule is used to acquire the quality level of the shooting location output by the scene understanding model based on the first text information;

[0046] The reporting maintenance submodule is used to report maintenance information for the shooting location when the quality level is lower than the quality level threshold.

[0047] In one embodiment, the video processing apparatus further includes:

[0048] The intelligent analysis module is used to obtain the intelligent analysis results of the target intelligent algorithm on the video to be detected; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0049] The intelligent analysis feedback module is used to input the fourth text information corresponding to the intelligent analysis result into the scene understanding model, and obtain the algorithm rule adjustment parameters corresponding to the algorithm type output by the scene understanding model;

[0050] The algorithm adjustment module is used to adjust the parameters according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and to load the adjusted target intelligent algorithm onto the video to be detected.

[0051] In one embodiment, the video processing apparatus further includes:

[0052] The camera control module is used to generate camera parameter adjustment information based on the comparison between the text description information and preset information, and the comparison between the statistical information in the intelligent analysis results and preset information, and to adjust the parameters of the subject being photographed based on the camera parameter adjustment information.

[0053] Thirdly, embodiments of this application provide an intelligent analysis system based on scene understanding, the intelligent analysis system comprising:

[0054] The video acquisition unit includes several shooting subjects deployed at different shooting locations;

[0055] The scene understanding unit includes a pre-trained image-to-text conversion model and a scene understanding model;

[0056] The image-to-text conversion model is used to receive the image to be detected captured by the subject and output the text description information of the image to be detected; wherein, the text description information describes the image to be detected in terms of image quality, image content and image elements.

[0057] The scene understanding model is used to receive the text description information and output algorithm adaptation information corresponding to the shooting location; wherein, the algorithm adaptation information includes at least the algorithm type;

[0058] The intelligent analysis unit includes an algorithm loading subunit, an intelligent analysis feedback subunit, and a camera control subunit;

[0059] The algorithm loading subunit is used to load the target intelligent algorithm onto the video to be detected taken by the subject at the shooting location according to the algorithm adaptation information, and to obtain the intelligent analysis results of the target intelligent algorithm on the video to be detected; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0060] The intelligent analysis feedback subunit is used to input the fourth text information corresponding to the intelligent analysis result into the scene understanding model;

[0061] The camera control subunit is used to generate camera parameter adjustment information based on the comparison between the text description information and preset information, and the comparison between the statistical information in the intelligent analysis results and preset information, and to adjust the parameters of the subject being photographed based on the camera parameter adjustment information.

[0062] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the video processing method described above.

[0063] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the video processing method described above.

[0064] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0065] This application provides a video processing method, apparatus, and intelligent analysis system based on scene understanding. The video processing method first acquires an image to be detected at a shooting location. Then, it uses a pre-trained image-to-text conversion model to obtain textual description information of the image to be detected. The textual description information describes the image quality, content, and elements of the image to be detected. This textual description information is then input into the pre-trained scene understanding model, which outputs algorithm adaptation information for the corresponding shooting location. Therefore, based on the obtained algorithm adaptation information, a target intelligent algorithm can be loaded onto the video to be detected captured by a shooting subject (e.g., a camera) positioned at that shooting location. This achieves automatic loading of the target intelligent algorithm without manual configuration, improving the deployment efficiency, configuration efficiency, and intelligent analysis quality of the shooting subject.

[0066] In other words, for different shooting locations, this embodiment of the application can sequentially obtain different algorithm adaptation information for each shooting location from different images to be detected at each shooting location through an image-to-text conversion model and a scene understanding model. Then, based on the algorithm adaptation information, the target intelligent algorithm can be loaded onto the video to be detected taken by the shooting subject at each shooting location, eliminating the need for manual configuration of the intelligent algorithm and improving the deployment efficiency, configuration efficiency, and intelligent analysis quality of the shooting subject. Attached Figure Description

[0067] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0068] Figure 1 is a flowchart illustrating the scene-understanding-based video processing method described in an embodiment of this application.

[0069] Figure 2 is a flowchart illustrating the scene-understanding-based video processing method described in one embodiment of this application.

[0070] Figure 3 is a flowchart illustrating the scene-understanding-based video processing method described in another embodiment of this application.

[0071] Figure 4 is a flowchart illustrating the scene-understanding-based video processing method described in another embodiment of this application.

[0072] Figure 5 is a flowchart illustrating the scene-understanding-based video processing method described in another embodiment of this application.

[0073] Figure 6 is a schematic diagram of the structure of the scene-understanding-based video processing device described in the embodiments of this application.

[0074] In the attached figures, the following labels are used: 100-Video acquisition unit, 101-Subject being captured, 102-Image to be detected, 103-Video to be detected, 200-Scene understanding unit, 201-Image-to-text conversion model, 202-Scene understanding model, 203-Text description information, 204-Algorithm adaptation information, 2011-Image quality evaluation sub-model, 2012-Image content extraction sub-model, 2013-Image element parsing sub-model, 2031-First text information, 2032-Second text information, 2033-Third text information, 2041-Algorithm rule adjustment parameters, 300-Intelligent analysis unit, 301-Algorithm loading sub-unit, 302-Intelligent analysis feedback sub-unit, 303-Camera control sub-unit, 3011-Intelligent analysis result, 3021-Fourth text information, 3031-Camera parameter adjustment information. 901 - Video shooting module, 902 - Image-to-text conversion module, 903 - Scene understanding module, 904 - Algorithm loading module. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments in this application are within the scope of protection of this application.

[0076] First, the technical terms used in the embodiments of this application will be explained:

[0077] Intelligent Analysis System: Composed of a camera and intelligent analysis algorithms. The intelligent analysis algorithms intelligently analyze the video stream captured by the camera, extracting structured information from the video stream and triggering alarms when specific behavioral events occur in the video stream.

[0078] Scene understanding: Performs advanced semantic information understanding on video images to assist in the deployment of intelligent analysis algorithms.

[0079] Shooting locations: The locations where cameras and other subjects are positioned for shooting.

[0080] Image-to-text conversion model: trained from text descriptions of each image. The text descriptions provide textual descriptions of the image from one or more perspectives.

[0081] Scene understanding models are trained using the user manuals for each intelligent algorithm. These manuals provide detailed explanations of the algorithms' business logic, application scenarios, application examples, location setup requirements, location selection specifications, rule area division, and rule parameter settings. Training the scene understanding model using these user manuals allows the knowledge of the algorithms' usage guidelines to be incorporated into the model, enabling it to serve intelligent analysis of video locations.

[0082] For each shooting location, the applicable intelligent algorithm varies depending on its location and shooting scene. Intelligent algorithms can include video structured analysis algorithms, perimeter security algorithms, behavior analysis algorithms, and smart city management algorithms, among others. These algorithms extract structured information from the video stream and trigger alarms based on specific rules regarding specific behaviors and times occurring within the stream. Different intelligent algorithms have different requirements regarding the setup of the shooting subject and the shooting scene.

[0083] Currently, it is necessary to manually configure intelligent algorithms for each shooting location and manually adjust the parameters of the intelligent algorithms. However, manually selecting intelligent algorithms involves a large amount of manual work, and construction personnel often lack the professional level, resulting in the shooting effect of the subjects at a large number of shooting locations being difficult to guarantee.

[0084] To address the above issues, this application provides a video processing method, apparatus, and intelligent analysis system based on scene understanding. First, textual description information of the image to be detected at the shooting location is obtained. Then, the textual description information is processed based on a scene understanding model to obtain algorithm adaptation information for the corresponding shooting location. In this way, a target intelligent algorithm is loaded according to the algorithm adaptation information to intelligently analyze the video to be detected captured by the shooting subject at the shooting location. This eliminates the need for manual configuration of the intelligent algorithm, improving the efficiency of shooting subject deployment, configuration, and the quality of intelligent analysis.

[0085] Figure 1 is a flowchart of the video processing method based on scene understanding, and Figure 2 is a flowchart of the video processing method based on scene understanding. The following explanation is based on Figure 1 and Figure 2.

[0086] The scene-understanding-based video processing method shown in Figure 1 includes:

[0087] S100: Obtain the image to be detected 102 at the shooting point. The image to be detected 102 is captured by the shooting subject 101 set up at the shooting point.

[0088] S200. Use a pre-trained image-to-text conversion model 201 to obtain text description information 203 of the image to be detected 102; wherein, the text description information 203 describes the image to be detected 102 in terms of image quality, image content and image elements.

[0089] S300, Input the text description information 203 into the pre-trained scene understanding model 202, and obtain the algorithm adaptation information 204 of the corresponding shooting location output by the scene understanding model 202; wherein, the algorithm adaptation information 204 includes at least the algorithm type.

[0090] S400. Based on the algorithm adaptation information 204, load the target intelligent algorithm onto the video 103 to be detected captured by the subject 101 at the shooting location.

[0091] Overall, for the image to be detected taken by the subject at the shooting location, a text-to-image conversion model is used to obtain the corresponding text description information. Then, a scene understanding model is used to obtain the algorithm adaptation information corresponding to the text description information. Finally, based on the algorithm adaptation information, the target intelligent algorithm is applied to the video to be detected taken by the subject at the shooting location.

[0092] In step S100, the image to be detected at the shooting point is first acquired.

[0093] A camera can be deployed at the designated shooting location. Then, the camera captures the image to be detected at that location. This image can be a frame from a video captured by the camera.

[0094] In step S200, after acquiring the image to be detected at the shooting location, a pre-trained image-to-text conversion model can be used to obtain the text description information corresponding to the image to be detected.

[0095] In one possible implementation, the image to be detected is input into a pre-trained image-to-text conversion model to obtain the text description information of the image to be detected output by the image-to-text conversion model.

[0096] As can be understood, the image-to-text conversion model in this embodiment, as described above, is obtained through pre-training. The input of the image-to-text conversion model is an image, and the output is a text description corresponding to that image.

[0097] Specifically, the text description information provides a textual description of the image to be detected from three aspects: image quality, image content, and image elements.

[0098] For example, the text description information corresponding to a certain image to be detected can be as follows:

[0099] Regarding image quality: the image to be tested has moderate brightness and color, no obstructions, and good clarity;

[0100] Regarding the image content: The image to be detected shows the situation at a traffic intersection, including a crossroads with vehicles traveling on the road;

[0101] Regarding image elements: the upper region of the image to be detected is the sky, which occupies 20% of the image; the middle region is the road, which occupies 50% of the image; and the two side regions are buildings, which occupy 30% of the image.

[0102] In step S300, after obtaining the text description information of the image to be detected, the pre-trained scene understanding model is used to obtain the algorithm adaptation information corresponding to the shooting location.

[0103] In one possible implementation, the textual description information of the image to be detected is input into a pre-trained scene understanding model to obtain the algorithm adaptation information output by the scene understanding model corresponding to the image to be detected (that is, corresponding to the shooting location).

[0104] As described above, the scene understanding model in this embodiment is pre-trained. The input to the scene understanding model is a text description, and the output is algorithm adaptation information corresponding to that text description. This algorithm adaptation information should at least include the algorithm type.

[0105] Continuing with the example above, after inputting the text description information into the scene understanding model, the scene understanding model outputs the algorithm adaptation information for the corresponding traffic intersection, such as the red light violation capture algorithm.

[0106] In step S400, after obtaining the algorithm adaptation information, the target intelligent algorithm can be loaded onto the video to be detected captured by the subject at the shooting point based on the algorithm adaptation information.

[0107] Continuing with the example above, the target intelligent algorithm is now a red light violation detection algorithm. That is, the red light violation detection algorithm is applied to the video to be detected captured by the camera at the shooting point located at the traffic intersection.

[0108] It is understandable that when cameras are deployed in public places such as waiting rooms, the images of the subjects to be detected, captured in the waiting room, are input into the image-to-text conversion model. Then, the text description information output by the image-to-text conversion model is input into the scene understanding model. Thus, the scene understanding model outputs a target intelligent algorithm applicable to the waiting room scenario. For example, a pedestrian flow monitoring algorithm, and / or a crowd gathering monitoring algorithm, etc.

[0109] It should be noted that the aforementioned target intelligent algorithm may be one or multiple.

[0110] As can be seen, in this embodiment, it is first necessary to obtain the textual description information of the image to be detected. Then, the textual description information is input into the scene understanding model to obtain the target intelligent algorithm applicable to the shooting location. In this way, by extracting high-level semantic information description of the image scene and utilizing the generality, keywords, and other characteristics of textual description, the target intelligent algorithm can be easily located using the algorithm adaptation information output by the scene understanding model, thus improving the matching effect of the intelligent algorithm.

[0111] This application provides a video processing method, apparatus, and intelligent analysis system based on scene understanding. First, an image to be detected at a shooting location is acquired. Then, a pre-trained image-to-text conversion model is used to obtain textual description information of the image to be detected. The textual description information describes the image quality, content, and elements of the image to be detected. This textual description information is then input into the pre-trained scene understanding model, which outputs algorithm adaptation information corresponding to the shooting location. Therefore, based on the obtained algorithm adaptation information, a target intelligent algorithm is loaded onto the video to be detected captured by a shooting subject (e.g., a camera) deployed at that shooting location. This achieves automatic loading of the target intelligent algorithm, eliminating the need for manual configuration and improving the deployment efficiency, configuration efficiency, and intelligent analysis quality of the shooting subject.

[0112] In other words, for different shooting locations, this embodiment of the application can sequentially obtain different algorithm adaptation information for each shooting location from different images to be detected at each shooting location through an image-to-text conversion model and a scene understanding model. Then, based on the algorithm adaptation information, the corresponding target intelligent algorithm is loaded onto the video to be detected captured by the shooting subject at each shooting location. This eliminates the need for manual configuration of intelligent algorithms, improving the deployment efficiency, configuration efficiency, and intelligent analysis quality of the shooting subject.

[0113] Regarding the image-text conversion model 201, in one possible implementation, the image-text conversion model 201 can be a holistic model.

[0114] Referring to Figure 3, in other possible implementations, the image-to-text conversion model 201 may include three specific sub-models: a pre-trained image quality evaluation sub-model 2011, an image content extraction sub-model 2012, and an image element parsing sub-model 2013.

[0115] Step S200 includes:

[0116] S201. Input the image to be detected 102 into the image quality evaluation sub-model 2011, the image content extraction sub-model 2012, and the image element parsing sub-model 2013 respectively, and obtain the first text information 2031 output by the image quality evaluation sub-model 2011, the second text information 2032 output by the image content extraction sub-model 2012, and the third text information 2033 output by the image element parsing sub-model 2013 respectively; wherein, the first text information 2031 reflects the text description of image quality, the second text information 2032 reflects the text description of image content, and the third text information 2033 reflects the text description of image elements.

[0117] That is, regarding the description of the image quality, image content, and image elements of the image to be detected mentioned above, this embodiment can be implemented using three specific sub-models respectively.

[0118] Specifically, the image to be detected is input into the image quality assessment sub-model, which outputs first text information. This first text information is a textual description of the image quality of the image to be detected. The image to be detected is input into the image content extraction sub-model, which outputs second text information. This second text information is a textual description of the image content of the image to be detected. The image to be detected is input into the image element parsing sub-model, which outputs third text information. This third text information is a textual description of the image elements of the image to be detected.

[0119] Continuing with the example above, the first text information includes the following description: The image to be detected has moderate brightness, moderate color, no occlusion, and good clarity.

[0120] The second text information includes the following description: The image to be detected shows the situation at a traffic intersection. The image includes a crossroads and vehicles traveling on the road.

[0121] The third text information includes the following description: The upper region of the image to be detected is the sky, which occupies 20% of the image; the middle region is the road, which occupies 50% of the image; and the two side regions are buildings, which occupy 30% of the image.

[0122] In other words, for the image to be detected, this embodiment can specifically use three sub-models (image quality evaluation sub-model, image content extraction sub-model, and image element parsing sub-model) to obtain three text information (first text information, second text information, and third text information) from three aspects (image quality, image content, and image elements). In this way, the three sub-models are trained separately, which improves the accuracy of text description.

[0123] The first text information 2031 describes the image to be detected in terms of sharpness, brightness, chroma, and occlusion. Depending on the actual needs, the first text information may also describe the image to be detected in terms of noise interference. In other words, the first text information can describe the image quality of the image to be detected from multiple dimensions. These multiple dimensions include, but are not limited to, sharpness, brightness, chroma, noise interference, and occlusion; this embodiment does not impose any limitations on these dimensions.

[0124] The second text information 2032 describes the objects in the image to be detected, their attributes, and the positional relationships between them. The third text information 2033 describes the object categories in the image to be detected, the location of each category of objects within the image, and the proportion of each category of objects in the image to be detected.

[0125] In one possible implementation, continuing to refer to FIG3, step S300 includes:

[0126] S301. Input the first text information 2031, the second text information 2032 and the third text information 2033 into the scene understanding model 202 simultaneously;

[0127] S304. Obtain the algorithm adaptation information 204 output by the scene understanding model 202, wherein the algorithm adaptation information 204 includes the algorithm type of the corresponding shooting point, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

[0128] After obtaining three textual information entries for the image to be detected through three sub-models, these three textual entries can be simultaneously input into the scene understanding model. The scene understanding model can then output algorithm adaptation information based on the textual descriptions of the three entries. This algorithm adaptation information should include the algorithm type, the loading region for the algorithm type, and the original parameters of the algorithm rules for the algorithm type.

[0129] Continuing with the example above, in the algorithm adaptation information output by the scene understanding model based on the textual description of the example above, the algorithm type is a red light violation capture algorithm, the loading area of ​​the algorithm type is the road area in the image to be detected, and the original parameters of the algorithm rule of the algorithm type are a general adaptation parameter or original parameter of this red light violation capture algorithm.

[0130] In one specific embodiment, step S300 further includes:

[0131] S302, Obtain the quality level of the shooting location output by the scene understanding model 202 based on the first text information 2031;

[0132] S303. When the quality level is lower than the quality level threshold, report the maintenance information of the shooting location.

[0133] The quality levels of shooting locations are categorized as: high quality, medium quality, and low quality.

[0134] Low-quality shooting locations need to be reported for maintenance. Videos captured by high-quality shooting locations are used for intelligent analysis.

[0135] For medium-quality shooting locations, the camera parameters of the corresponding subject are adjusted to improve the quality level of that location. Then, the quality level of each shooting location is reassessed after the adjustments. Since the camera parameters of the subject at the medium-quality shooting locations have been adjusted, the previously medium-quality shooting locations have now been upgraded to high-quality. Therefore, the reassessment results include both low-quality and high-quality shooting locations. Low-quality shooting locations need to be reported for maintenance. The videos captured at high-quality shooting locations are used for intelligent analysis.

[0136] Before outputting algorithm adaptation information, the scene understanding model can also output a quality level based on the first text information. This quality level reflects the shooting quality of the shooting point corresponding to the image to be detected. When the quality level is lower than a preset quality level threshold, that is, the quality level of the shooting point is low quality, it indicates that the image quality of the image to be detected acquired at that shooting point is poor and needs to be reported for maintenance.

[0137] For example, when the first text information of the image to be detected contains textual descriptions such as "severe occlusion", the scene understanding model can classify the shooting point corresponding to the image to be detected as a low quality level based on the keyword "severe occlusion" and report the maintenance information of the shooting point.

[0138] In one possible implementation, referring to FIG4, the video processing method further includes:

[0139] S500, Obtain the intelligent analysis results 3011 of the target intelligent algorithm on the video 103 to be detected; wherein, the intelligent analysis results 3011 include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0140] S600. Input the fourth text information 3021 corresponding to the intelligent analysis result 3011 into the scene understanding model 202, and obtain the algorithm rule adjustment parameter 2041 of the corresponding algorithm type output by the scene understanding model 202.

[0141] S700. Adjust parameter 2041 according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and load the adjusted target intelligent algorithm into the video to be detected.

[0142] Overall, this embodiment can adjust the algorithm rule parameters of the target intelligent algorithm based on the intelligent analysis results of the video to be detected, thereby obtaining an adjusted target intelligent algorithm. Using the adjusted target intelligent algorithm to perform intelligent analysis on the video to be detected can improve the quality of intelligent analysis of the target intelligent algorithm.

[0143] Specifically, the first step is to use a pre-defined target intelligent algorithm to intelligently analyze the video to be detected taken by the subject at the shooting location, and obtain the intelligent analysis results.

[0144] The intelligent analysis results include statistical information on target objects or events identified from the video to be detected using a target intelligence algorithm within a predetermined time period, under different dimensions.

[0145] For example, when the target intelligent algorithm is a red-light violation detection algorithm, the target object is a vehicle running a red light. In this case, the intelligent analysis results include: statistical information on vehicles running red lights identified by the red-light violation detection algorithm within a predetermined time period, across different dimensions. These different dimensions include: the size, mass, and orientation of the vehicle running the red light, as well as the positive report rate and recall rate.

[0146] For example, when the target intelligent algorithm is a crowd gathering monitoring algorithm, the target event is a crowd gathering event. In this case, the intelligent analysis results include: statistical information on the crowd gathering event detected by the crowd gathering monitoring algorithm within a predetermined time period, under different dimensions. These different dimensions include: the size, quality, orientation, positive report rate, and recall rate of the gathered crowd, etc.

[0147] Secondly, after obtaining the above intelligent analysis results, they are converted into textual descriptions, thus obtaining the fourth textual information. Then, this fourth textual information is input into the scene understanding model to obtain the algorithm rule adjustment parameters corresponding to the algorithm type output by the scene understanding model.

[0148] The subject continuously captures images, resulting in a continuous video for testing. A segment of this continuous video is selected. A target intelligence algorithm is then used to perform intelligent analysis on this selected segment, yielding the analysis results.

[0149] Then, based on the intelligent analysis results of the video segment to be detected, the algorithm rule adjustment parameters for the corresponding algorithm type are determined. According to the algorithm rule adjustment parameters for the algorithm type, the parameters of the target intelligent algorithm are adjusted to obtain the adjusted target intelligent algorithm.

[0150] The adjusted target intelligence algorithm has undergone parameter optimization. Using the adjusted algorithm, the video segment to be detected is analyzed again to improve the quality of the intelligent analysis. Alternatively, the adjusted algorithm can be used to analyze videos captured after this initial video segment to further improve the quality of the intelligent analysis.

[0151] In other words, the scenario understanding model processes the fourth text information obtained through intelligent analysis, outputs algorithm rules to adjust parameters, adjusts the parameters according to the algorithm rules, loads the target intelligent algorithm, optimizes the parameters of the target intelligent algorithm, and uses the adjusted target intelligent algorithm to analyze the video to be detected, thereby improving the quality of intelligent analysis.

[0152] In one possible implementation, referring to FIG5, the video processing method further includes:

[0153] S800: Based on the comparison between the text description information 203 and the preset information, and the comparison between the statistical information in the intelligent analysis result 3011 and the preset information, camera parameter adjustment information 3031 is generated, and the parameters of the subject 101 are adjusted according to the camera parameter adjustment information 3031.

[0154] After obtaining the above intelligent analysis results, in addition to optimizing the parameters of the target intelligent algorithm, the text description information can be compared with the preset information, and the statistical information in the intelligent analysis results can be compared with the preset information. Based on the comparison, the parameters of the shooting subject (i.e., the camera) can be adjusted to improve the shooting quality.

[0155] For example, taking the size dimension in the intelligent analysis results as an example, when the target intelligent algorithm is a red-light violation detection algorithm, and the target object is a vehicle running a red light, the intelligent analysis results include: statistical information on the size dimension of vehicles that have been detected by the red-light violation detection algorithm within a predetermined time period. Specifically, small-sized vehicles account for 65%, medium-sized vehicles account for 20%, and large-sized vehicles account for 15%. However, the preset information suggests that the proportion of medium-sized vehicles should be greater than 30%, and the proportion of large-sized vehicles should be greater than 50%.

[0156] Based on a comparison of the statistical information in the intelligent analysis results with the preset information, the video currently captured by the subject does not meet the preset requirements. Furthermore, this embodiment can also comprehensively consider the comparison of the text description information with the preset information to determine whether the video currently captured by the subject meets the preset requirements.

[0157] Therefore, this embodiment can generate camera parameter adjustment information based on a comparison between the text description information and preset information, as well as a comparison between the statistical information on size dimensions in the intelligent analysis results and the preset information. The camera parameter adjustment information can be the camera magnification, and the parameters of the subject being photographed are adjusted according to this camera magnification until the captured video meets the preset requirements.

[0158] Of course, the method of generating camera parameter adjustment information based on statistical information such as quality dimension and orientation dimension based on intelligent analysis results is similar to the method of generating camera parameter adjustment information based on statistical information in size dimension, and will not be described again.

[0159] Based on the above video processing method, this application also discloses a scene-understanding-based video processing apparatus. Referring to Figure 6, the video processing apparatus includes:

[0160] The video capture module 901 is used to acquire the image to be detected at the capture point. The image to be detected is captured by the subject set up at the capture point.

[0161] The image-to-text conversion module 902 is used to obtain textual description information of the image to be detected using a pre-trained image-to-text conversion model; wherein, the textual description information describes the image to be detected in terms of image quality, image content and image elements.

[0162] The scene understanding module 903 is used to input text description information into a pre-trained scene understanding model and obtain the algorithm adaptation information of the corresponding shooting location output by the scene understanding model; wherein, the algorithm adaptation information includes at least the algorithm type.

[0163] The algorithm loading module 904 is used to load the target intelligent algorithm onto the video to be detected taken by the subject at the shooting location based on the algorithm adaptation information.

[0164] In one possible implementation, the image-to-text conversion model includes a pre-trained image quality assessment sub-model, an image content extraction sub-model, and an image element parsing sub-model;

[0165] This image-to-text conversion module is specifically used for:

[0166] The image to be detected is input into the image quality assessment sub-model, the image content extraction sub-model, and the image element parsing sub-model, respectively, and the first text information output by the image quality assessment sub-model, the second text information output by the image content extraction sub-model, and the third text information output by the image element parsing sub-model are obtained respectively; wherein, the first text information reflects the text description of image quality, the second text information reflects the text description of image content, and the third text information reflects the text description of image elements.

[0167] In one possible implementation, the first text information describes the image quality of the image to be detected from multiple dimensions, including at least sharpness, brightness, chroma, and occlusion; the second text information describes the objects in the image to be detected, the attributes of the objects, and the positional relationships between the objects; and the third text information describes the object categories in the image to be detected, the location of each category of objects in the image, and the proportion of each category of objects in the image to be detected.

[0168] In one possible implementation, the scene understanding module includes:

[0169] The text input submodule is used to simultaneously input the first text information, the second text information, and the third text information into the scene understanding model;

[0170] The algorithm acquisition submodule is used to acquire the algorithm adaptation information output by the scene understanding model. The algorithm adaptation information includes the algorithm type of the corresponding shooting point, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

[0171] In one possible implementation, the scene understanding module further includes:

[0172] The quality level acquisition submodule is used to acquire the quality level of the shooting location output by the scene understanding model based on the first text information;

[0173] The reporting and maintenance submodule is used to report maintenance information for shooting locations when the quality level is lower than the quality level threshold.

[0174] In one possible implementation, the video processing apparatus further includes:

[0175] The intelligent analysis module is used to obtain the intelligent analysis results of the target intelligent algorithm on the video to be detected; the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0176] The intelligent analysis feedback module is used to input the fourth text information corresponding to the intelligent analysis results into the scene understanding model and obtain the algorithm rule adjustment parameters of the corresponding algorithm type output by the scene understanding model.

[0177] The algorithm adjustment module is used to adjust parameters according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and load the adjusted target intelligent algorithm onto the video to be detected.

[0178] In one possible implementation, the video processing apparatus further includes:

[0179] The camera control module is used to generate camera parameter adjustment information based on the comparison between text description information and preset information, as well as the comparison between statistical information in the intelligent analysis results and preset information, and to adjust the parameters of the subject being photographed based on the camera parameter adjustment information.

[0180] Based on the video processing method described above, this application also discloses an intelligent analysis system based on scene understanding. Referring to Figure 5, this intelligent analysis system includes:

[0181] The video acquisition unit 100 includes several shooting subjects 101 set up at different shooting locations;

[0182] Scene understanding unit 200 includes a pre-trained image-to-text conversion model 201 and a scene understanding model 202;

[0183] The image-to-text conversion model 201 is used to receive the image to be detected 102 captured by the shooting subject 101 and output the text description information 203 of the image to be detected 102; wherein, the text description information 203 describes the image to be detected 102 in terms of image quality, image content and image elements.

[0184] The scene understanding model 202 is used to receive text description information 203 and output algorithm adaptation information 204 corresponding to the shooting location; wherein, the algorithm adaptation information 204 includes at least the algorithm type;

[0185] The intelligent analysis unit 300 includes an algorithm loading subunit 301, an intelligent analysis feedback subunit 302, and a camera control subunit 303.

[0186] The algorithm loading subunit 301 is used to load the target intelligent algorithm onto the video 103 to be detected captured by the shooting subject 101 at the shooting point according to the algorithm adaptation information 204, and obtain the intelligent analysis result 3011 of the target intelligent algorithm on the video 103 to be detected; wherein, the intelligent analysis result 3011 includes: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions;

[0187] The intelligent analysis feedback subunit 302 is used to input the fourth text information 3021 corresponding to the intelligent analysis result 3011 into the scene understanding model 202;

[0188] The camera control subunit 303 is used to generate camera parameter adjustment information 3031 based on the comparison between the text description information 203 and the preset information, and the comparison between the statistical information in the intelligent analysis result 3011 and the preset information, and to adjust the parameters of the subject 101 based on the camera parameter adjustment information 3031.

[0189] This application also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the video processing method described above.

[0190] This application also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the video processing method described above.

[0191] This application also discloses an electronic device, comprising:

[0192] Memory, used to store computer programs;

[0193] The processor, when executing the program stored in the memory, implements the above video processing method.

[0194] Furthermore, the aforementioned electronic devices may also include a communication bus and / or a communication interface, with the processor, communication interface, and memory communicating with each other via the communication bus.

[0195] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0196] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0197] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0198] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0199] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0200] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0201] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0202] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0203] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize that certain variations, modifications, alterations, additions, and sub-combinations thereof should be included within the scope of protection of this application.

Claims

1. A video processing method based on scene understanding, characterized in that, The video processing method includes: The image to be detected is obtained at the shooting point, wherein the image to be detected is captured by the shooting subject deployed at the shooting point; A pre-trained image-to-text conversion model is used to obtain textual description information of the image to be detected; wherein, the textual description information describes the image to be detected in terms of image quality, image content, and image elements. The text description information is input into a pre-trained scene understanding model, and the algorithm adaptation information corresponding to the shooting location is obtained from the scene understanding model; wherein, the algorithm adaptation information includes at least the algorithm type; Based on the algorithm adaptation information, the target intelligent algorithm is loaded onto the video to be detected captured by the subject at the shooting location.

2. The video processing method according to claim 1, characterized in that, The image-to-text conversion model includes a pre-trained image quality assessment sub-model, an image content extraction sub-model, and an image element parsing sub-model. The step of obtaining the text description information of the image to be detected using a pre-trained image-to-text conversion model includes: The image to be detected is input into the image quality assessment sub-model, the image content extraction sub-model, and the image element parsing sub-model, respectively, and the first text information output by the image quality assessment sub-model, the second text information output by the image content extraction sub-model, and the third text information output by the image element parsing sub-model are obtained respectively; wherein, the first text information reflects the textual description of the image quality, the second text information reflects the textual description of the image content, and the third text information reflects the textual description of the image elements.

3. The video processing method according to claim 2, characterized in that, The first text information describes the image quality of the image to be detected from multiple dimensions, including at least sharpness, brightness, chroma and occlusion. The second text information describes the objects in the image to be detected, their attributes, and the positional relationships between them. The third text information describes the objects in the image to be detected in terms of their categories, the location of each category of objects in the image, and the proportion of each category of objects in the image to be detected.

4. The video processing method according to claim 2, characterized in that, The step of inputting the text description information into a pre-trained scene understanding model and obtaining the algorithm adaptation information corresponding to the shooting location output by the scene understanding model includes: The first text information, the second text information, and the third text information are simultaneously input into the scene understanding model; Obtain the algorithm adaptation information output by the scene understanding model, wherein the algorithm adaptation information includes the algorithm type corresponding to the shooting location, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

5. The video processing method according to claim 4, characterized in that, Before the step of obtaining the algorithm adaptation information output by the scene understanding model, the method further includes: Obtain the quality level of the shooting location output by the scene understanding model based on the first text information; If the quality level is lower than the quality level threshold, a maintenance message for the shooting location is reported.

6. The video processing method according to claim 4, characterized in that, After the step of loading the target intelligent algorithm onto the video to be detected captured by the subject at the shooting location based on the algorithm adaptation information, the video processing method further includes: The intelligent analysis results of the target intelligent algorithm on the video to be detected are obtained; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions; The fourth text information corresponding to the intelligent analysis result is input into the scene understanding model, and the algorithm rule adjustment parameters corresponding to the algorithm type are output by the scene understanding model. The parameters are adjusted according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and the adjusted target intelligent algorithm is loaded onto the video to be detected.

7. The video processing method according to claim 6, characterized in that, After obtaining the intelligent analysis results of the target intelligent algorithm on the video to be detected, the video processing method further includes: Based on the comparison between the text description information and the preset information, and the comparison between the statistical information in the intelligent analysis results and the preset information, camera parameter adjustment information is generated, and the parameters of the subject being photographed are adjusted according to the camera parameter adjustment information.

8. A video processing device based on scene understanding, characterized in that, The video processing device includes: The video shooting module is used to acquire the image to be detected at the shooting point, wherein the image to be detected is captured by the shooting subject deployed at the shooting point; The image-to-text conversion module is used to obtain textual description information of the image to be detected using a pre-trained image-to-text conversion model; wherein, the textual description information describes the image to be detected in terms of image quality, image content, and image elements. The scene understanding module is used to input the text description information into a pre-trained scene understanding model and obtain the algorithm adaptation information corresponding to the shooting location output by the scene understanding model; wherein, the algorithm adaptation information includes at least the algorithm type; The algorithm loading module is used to load the target intelligent algorithm onto the video to be detected captured by the subject at the shooting location based on the algorithm adaptation information.

9. The video processing apparatus according to claim 8, characterized in that, The image-to-text conversion model includes a pre-trained image quality assessment sub-model, an image content extraction sub-model, and an image element parsing sub-model. The image-to-text conversion module is specifically used for: The image to be detected is input into the image quality assessment sub-model, the image content extraction sub-model, and the image element parsing sub-model, respectively, and the first text information output by the image quality assessment sub-model, the second text information output by the image content extraction sub-model, and the third text information output by the image element parsing sub-model are obtained respectively; wherein, the first text information reflects the textual description of the image quality, the second text information reflects the textual description of the image content, and the third text information reflects the textual description of the image elements.

10. The video processing apparatus according to claim 9, characterized in that, The first text information describes the image quality of the image to be detected from multiple dimensions, including at least sharpness, brightness, chroma and occlusion. The second text information describes the objects in the image to be detected, their attributes, and the positional relationships between them. The third text information describes the objects in the image to be detected in terms of their categories, the location of each category of objects in the image, and the proportion of each category of objects in the image to be detected.

11. The video processing apparatus according to claim 9, characterized in that, The scene understanding module includes: The text input submodule is used to simultaneously input the first text information, the second text information, and the third text information into the scene understanding model; The algorithm acquisition submodule is used to acquire the algorithm adaptation information output by the scene understanding model, wherein the algorithm adaptation information includes the algorithm type corresponding to the shooting point, the loading area of ​​the algorithm type, and the original parameters of the algorithm rule of the algorithm type.

12. The video processing apparatus according to claim 11, characterized in that, The scene understanding module also includes: The quality level acquisition submodule is used to acquire the quality level of the shooting location output by the scene understanding model based on the first text information; The reporting maintenance submodule is used to report maintenance information for the shooting location when the quality level is lower than the quality level threshold.

13. The video processing apparatus according to claim 11, characterized in that, The video processing device further includes: The intelligent analysis module is used to obtain the intelligent analysis results of the target intelligent algorithm on the video to be detected; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions; The intelligent analysis feedback module is used to input the fourth text information corresponding to the intelligent analysis result into the scene understanding model, and obtain the algorithm rule adjustment parameters corresponding to the algorithm type output by the scene understanding model; The algorithm adjustment module is used to adjust the parameters according to the algorithm rules of the algorithm type to obtain the adjusted target intelligent algorithm, and to load the adjusted target intelligent algorithm onto the video to be detected.

14. The video processing apparatus according to claim 13, characterized in that, The video processing device further includes: The camera control module is used to generate camera parameter adjustment information based on the comparison between the text description information and preset information, and the comparison between the statistical information in the intelligent analysis results and preset information, and to adjust the parameters of the subject being photographed based on the camera parameter adjustment information.

15. An intelligent analysis system based on scene understanding, characterized in that, The intelligent analysis system includes: The video acquisition unit includes several shooting subjects deployed at different shooting locations; The scene understanding unit includes a pre-trained image-to-text conversion model and a scene understanding model; The image-to-text conversion model is used to receive the image to be detected captured by the subject and output the text description information of the image to be detected; wherein, the text description information describes the image to be detected in terms of image quality, image content and image elements. The scene understanding model is used to receive the text description information and output algorithm adaptation information corresponding to the shooting location; wherein, the algorithm adaptation information includes at least the algorithm type; The intelligent analysis unit includes an algorithm loading subunit, an intelligent analysis feedback subunit, and a camera control subunit; The algorithm loading subunit is used to load the target intelligent algorithm onto the video to be detected captured by the subject at the shooting location according to the algorithm adaptation information, and to obtain the intelligent analysis results of the target intelligent algorithm on the video to be detected; wherein, the intelligent analysis results include: statistical information of the target object or target event identified by the target intelligent algorithm in different dimensions; The intelligent analysis feedback subunit is used to input the fourth text information corresponding to the intelligent analysis result into the scene understanding model; The camera control subunit is used to generate camera parameter adjustment information based on the comparison between the text description information and preset information, and the comparison between the statistical information in the intelligent analysis results and preset information, and to adjust the parameters of the subject being photographed based on the camera parameter adjustment information.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the video processing method as described in any one of claims 1-7.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the video processing method as described in any one of claims 1-7.

Images