Image information analysis method, device, electronic device and system

Abstract

The invention relates to an image information analysis method and device, an electronic device, a storage medium and a system.The image information analysis method comprises the steps of obtaining a to-be-analyzed video image and a behavior rule description instruction, inputting the to-be-analyzed video image and the behavior rule description instruction into a rule analysis large model, and responding to the behavior rule description instruction, determining rule direction information corresponding to the to-be-analyzed video image according to semantic description information contained in the behavior rule description instruction, extracting a feature vector of the to-be-analyzed video image, determining rule coordinate information corresponding to the to-be-analyzed video image according to the feature vector, and determining the to-be-analyzed video image according to the rule direction information and the rule coordinate information. And determining behavior rule information corresponding to the to-be-analyzed video image, and outputting the behavior rule information by the rule analysis large model. Through the method and the device, the problem of how to adaptively generate the image behavior analysis rule in related technologies is solved, rule generation is automatic and low in cost, and rule information is high in accuracy.

Description

Technical Field

[0001] The present application relates to the field of video image processing, and in particular to an image information analysis method, device, electronic device, storage medium and system. Background Art

[0002] The core of general behavior analysis intelligent algorithms is to understand and analyze complex behavior patterns, which are often hidden in large amounts of data. In the field of image behavior analysis, rule models such as rule lines in images can be manually set so that the algorithm can define and identify behavior patterns in images based on corresponding rules. For example, by setting a series of conditions and thresholds, the algorithm can identify specific behaviors or events.

[0003] However, the rule requirements are different in different scenes. For fixed scenes, they usually do not need to be maintained after being manually drawn once. But for non-fixed scenes such as ball cameras, different rules need to be drawn for different scenes. The current method is to set preset points for ball cameras or pre-build additional facilities to mark points, and manually draw different rules at different points. This method has the problem of high drawing difficulty or high maintenance cost, and the scene at non-preset points cannot be monitored for behavior. In addition, for winding boundaries such as the coast or lakeside, the boundary rule line can often be simply drawn with more than two points, and the manual drawing workload is large and the maintenance cost is high. Moreover, at high tide and low tide, the boundary line of the scene picture changes, and manual drawing cannot be accurately and timely maintained. It can only roughly determine different rules for a certain time period. Therefore, the adaptability of the rule generation method in the related art is limited, and it cannot cope with the complexity and dynamic changes of image data. In addition, rule setting, maintenance and updating all require manual intervention, which is low in accuracy and high in cost.

[0004] Currently, no effective solution has been proposed for the problem of how to adaptively generate image behavior analysis rules in related technologies. Summary of the invention

[0005] The embodiments of the present application provide an image information analysis method, device, electronic device, storage medium and system to at least solve the problem of how to adaptively generate image behavior analysis rules in the related art.

[0006] In a first aspect, an embodiment of the present application provides an image information analysis method.

[0007] In some embodiments, the image information analysis method includes:

[0008] Obtaining a video image to be analyzed and a behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into a rule analysis big model;

[0009] In response to the behavior rule description instruction, determining the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction;

[0010] In response to the behavior rule description instruction, extracting a feature vector of the video image to be analyzed, and determining rule coordinate information corresponding to the video image to be analyzed according to the feature vector;

[0011] The behavior rule information corresponding to the video image to be analyzed is determined according to the rule direction information and the rule coordinate information, and the rule analysis model outputs the behavior rule information.

[0012] In some embodiments, after determining the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, the rule analysis model outputs the behavior rule information and further includes:

[0013] Sending the regular direction information and the regular coordinate information to a client device;

[0014] In response to the rule information analysis instruction from the client device, the rule direction information and the rule coordinate information are sent to the server device.

[0015] In some embodiments, after the rule information analysis instruction from the client device is sent to the server device in response to the rule information analysis instruction, the following step further includes:

[0016] According to the regular coordinate information, the server device determines the target boundary line and / or target boundary area in the video image to be analyzed.

[0017] In some embodiments, it is characterized in that the obtaining of the video image to be analyzed and the behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into the rule analysis big model comprises:

[0018] Acquire a video image to be analyzed and a behavior rule description instruction, and determine a video capture device corresponding to the video image to be analyzed;

[0019] In the case that the video shooting device is a non-fixed shooting device and the position of the non-fixed shooting device is updated, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis large model.

[0020] In some embodiments, after obtaining the video image to be analyzed and the behavior rule description instruction, and determining the video shooting device corresponding to the video image to be analyzed, the method further includes:

[0021] When the video capture device is a fixed capture device, or when the video capture device is a non-fixed capture device and the position of the non-fixed capture device is not updated, determining current boundary information in the video image to be analyzed;

[0022] Calculating a repetition weight between the current boundary information and the preset boundary information;

[0023] When the repetition weight meets a preset threshold condition, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis model.

[0024] In some embodiments, the rule analysis model includes a semantic understanding module and a multimodal analysis module, and the behavior rule information output by the rule analysis model includes:

[0025] In response to the behavior rule description instruction, the semantic understanding module outputs a semantic parsing result, and the multimodal analysis module outputs a multimodal analysis result;

[0026] According to the semantic parsing result and the multimodal analysis result, the behavior rule information corresponding to the video image to be analyzed is determined.

[0027] In a second aspect, an embodiment of the present application provides an image information analysis device.

[0028] In some embodiments, the image information analysis device includes an analysis object input module, a direction information determination module, a coordinate information determination module and a rule information output module:

[0029] The analysis object input module is used to obtain the video image to be analyzed and the behavior rule description instruction, and input the video image to be analyzed and the behavior rule description instruction into the rule analysis big model;

[0030] The direction information determination module is used to respond to the behavior rule description instruction and determine the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction;

[0031] The coordinate information determination module is used to extract the feature vector of the video image to be analyzed in response to the behavior rule description instruction, and determine the rule coordinate information corresponding to the video image to be analyzed according to the feature vector;

[0032] The rule information output module is used to determine the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, so that the rule analysis model outputs the behavior rule information.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image information analysis method as described in the first aspect above is implemented.

[0034] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the image information analysis method as described in the first aspect above.

[0035] In a fifth aspect, an embodiment of the present application provides an image information analysis system, the system comprising: a terminal device, a transmission device, and a server device; wherein the terminal device is connected to the server device via the transmission device;

[0036] The terminal device is used to receive information sent by the server device and / or information input by the user, and display information;

[0037] The transmission device is used to transmit information;

[0038] The server device is used to execute the image information analysis method as described in the first aspect above.

[0039] Compared with the related art, the image information analysis method, device, electronic device, storage medium and system provided in the embodiments of the present application obtain the video image to be analyzed and the behavior rule description instruction, and input the video image to be analyzed and the behavior rule description instruction into the rule analysis big model; in response to the behavior rule description instruction, determine the rule direction information corresponding to the video image to be analyzed according to the semantic description information, extract the feature vector of the video image to be analyzed, determine the rule coordinate information corresponding to the video image to be analyzed according to the feature vector, and further determine the behavior rule information corresponding to the video image to be analyzed, and the rule analysis big model outputs the behavior rule information, which solves the problem of how to adaptively generate image behavior analysis rules in the related art, and the rule generation is automated and low-cost, and the rule information is highly accurate.

[0040] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1is a hardware structure block diagram of a terminal according to an image information analysis method of an embodiment of the present application;

[0043] Figure 2 is a flowchart of an image information analysis method according to an embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a video image to be analyzed containing behavior rule information according to an embodiment of the present application;

[0045] Figure 4 is a flow chart of an image information analysis method according to a preferred embodiment of the present application;

[0046] Figure 5 It is a structural block diagram of an image information analysis device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0048] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the image information analysis method according to an embodiment of the present invention. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image information analysis method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] The present application embodiment provides an image information analysis method, Figure 2 is a flowchart of an image information analysis method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0054] Step S201, obtaining the video image to be analyzed and the behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into the rule analysis big model.

[0055] In an embodiment of the present application, the surveillance video data captured by the front-end camera can be obtained, and the video image therein can be used as the video image to be analyzed, and the video image to be analyzed can include multiple frames of real-time images. The behavior rule description instruction can be an instruction to set the scene through a text description provided by the user. The video image to be analyzed and the behavior rule description instruction are input into the rule analysis model, wherein the rule analysis model can be a model trained based on a model with strong language understanding and generation capabilities through large-scale text data training.

[0056] Step S202, in response to the behavior rule description instruction, determining the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction.

[0057] In response to the behavior rule description instruction, the rule analysis model can understand and parse the semantic information in the instruction text and extract hidden rules and patterns from the unstructured text. The behavior rule description instruction includes semantic description information, and the rule direction information corresponding to the video image to be analyzed is determined based on the semantic description information and the scene of semantic understanding.

[0058] Step S203 , in response to the behavior rule description instruction, extract the feature vector of the video image to be analyzed, and determine the rule coordinate information corresponding to the video image to be analyzed according to the feature vector.

[0059] In some embodiments, the rule analysis model includes a semantic understanding module, which obtains the I frame of the video image frame and gives it to the rule analysis model for analysis, extracts the feature vector, and matches it with the sample feature vector in the semantic understanding module to determine multiple boundary point information, thereby determining the rule coordinate information corresponding to the video image to be analyzed.

[0060] Step S204, determining the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, and the rule analysis model outputs the behavior rule information.

[0061] Among them, the behavior rule information corresponding to the video image to be analyzed may include rule direction information and rule coordinate information, and the rule analysis big model outputs the rule direction information and rule coordinate information for subsequent processing. When the big model is applied to behavior analysis, the text describing the behavior rules can be input, and the big model can learn the logic of the rules by understanding the semantics of the text, automatically "draw" the rule lines, and output the behavior rule information corresponding to the video image to be analyzed, including the rule direction information and rule coordinate information, which can be further converted into a rule model that the algorithm can understand.

[0062] Through the above steps, semantic analysis is performed based on the rule analysis big model, and the text information in the behavior rule description instructions acting on the video image to be analyzed is converted into a deep understanding. At the same time, multimodal analysis is performed on the real-time video stream, and the behavior rule information corresponding to the video image to be analyzed is automatically generated according to the analyzed scene information. Specifically, it may include rule direction information determined according to the semantic description information, and rule coordinate information determined according to the feature vector, so as to adaptively improve the accuracy and efficiency of image analysis, so that the output behavior rule information has high accuracy and low cost.

[0063] In some embodiments, after step S204, the method further includes:

[0064] Step S205: sending the regular direction information and the regular coordinate information to the client device.

[0065] Furthermore, the output rule direction information and rule coordinate information will be sent to the client device and displayed visually to the user, allowing the user to perform secondary editing and confirmation.

[0066] Step S206, in response to the rule information analysis instruction from the client device, the rule direction information and the rule coordinate information are sent to the server device.

[0067] When a user initiates a rule information analysis instruction, the rule direction information and rule coordinate information will be sent to the algorithm system of the server device for further intelligent analysis and processing. The rule information analysis instruction may be a rule update instruction.

[0068] Through the above steps, the embodiment of the present application sends the rule direction information and rule coordinate information to the client device, allowing the user to perform secondary confirmation, improving the accuracy and flexibility of the rules, while increasing visualization and user interactivity.

[0069] In some embodiments, after step S206, the method further includes:

[0070] Step S207: The server device determines the target boundary line and / or target boundary area in the video image to be analyzed according to the regular coordinate information.

[0071] The rule analysis model outputs rule coordinate information based on the parsed scene information. Further, the server device can generate boundary coordinates of a regular shape based on the rule coordinate information and draw it into a target boundary line and / or target boundary area, such as drawing a boundary line segment or enclosing an area (such as a quadrilateral), etc. It is worth mentioning that a rule direction diagram can also be drawn based on the rule direction information, such as a direction diagram of crossing the boundary line rule in a video image. Figure 3 is a schematic diagram of a video image to be analyzed containing behavior rule information according to an embodiment of the present application. Figure 3 As shown, the scene of the video image to be analyzed is line crossing and height climbing detection, and the rule coordinate information includes coordinate point A and coordinate point B, and a line segment can be drawn according to coordinate point A and coordinate point B. The rule direction information includes the line crossing rule direction, as shown by the arrow direction in the figure.

[0072] Through the above steps, the behavior rule information generated by the embodiment of the present application can be connected to the traditional algorithm system of the server-side device, and generate the target boundary line and / or target boundary area in the video image to be analyzed, thereby realizing the integration of technologies and improving the efficiency of the overall system.

[0073] In some embodiments, step S201 includes:

[0074] Step S2011, obtaining the video image to be analyzed and the behavior rule description instruction, and determining the video shooting device corresponding to the video image to be analyzed.

[0075] In addition to obtaining the video image to be analyzed and the behavior rule description instructions, it is also necessary to determine whether the video image data to be analyzed is from a fixed scene or a variable scene by determining the video shooting device corresponding to the video image to be analyzed.

[0076] Step S2012: When the video shooting device is a non-fixed shooting device and the position of the non-fixed shooting device is updated, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis large model.

[0077] When the video capture device is a non-fixed capture device, such as a dome camera, the corresponding scene is determined to be a variable scene. In a variable scene, if the position of the non-fixed capture device is updated, such as the dome camera position changes, the video image to be analyzed and the behavior rule description instructions are input into the rule analysis big model, and the big model semantic scene understanding and the latest video frame data are obtained for analysis to output the latest behavior rule information, such as coordinates for boundary drawing and other data.

[0078] Through the above steps, when the scene picture changes dynamically and the rule lines or rule areas also change accordingly, the embodiment of the present application does not need to manually monitor the picture changes or pre-set dynamic rules. Instead, after determining the non-fixed scene, the rule analysis large model adaptively adjusts the behavior rule information according to the dynamic changes, which can better understand and parse complex scene requirements and improve the accuracy of the analysis.

[0079] In some embodiments, after step S2011, the method further includes:

[0080] S2013: When the video capture device is a fixed capture device, or when the video capture device is a non-fixed capture device and the position of the non-fixed capture device is not updated, determine current boundary information in the video image to be analyzed.

[0081] When the video capture device is a fixed capture device, or when the video capture device is a non-fixed capture device and the position of the non-fixed capture device is not updated, the corresponding scene is determined to be a fixed scene. For example, when the video capture device is a gun camera, the corresponding scene is determined to be a fixed scene. When the video capture device is a ball camera and the position of the ball camera has not changed, the corresponding scene is determined to be a fixed scene. When the corresponding scene is determined to be a fixed scene, the current boundary information in the video image to be analyzed is determined.

[0082] S2014, calculating the repetition weight between the current boundary information and the preset boundary information.

[0083] In the case of a fixed scene, the image information analysis method of the embodiment of the present application can monitor the scene periodically, and the initialization boundary information can be a boundaryless record, and the boundary information corresponding to the cycle is determined in each monitoring cycle. In this way, the preset boundary information can be the boundary information determined in the previous cycle of the cycle in which the video image to be analyzed is located. The repetition weight between the current boundary information and the preset boundary information can be the similarity between the two based on a percentage measurement.

[0084] S2015: When the repetition weight meets the preset threshold condition, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis model.

[0085] In accordance with the above, the repetition weight may be the similarity between the current boundary information and the preset boundary information. The preset threshold condition may be that the repetition weight is less than the preset weight threshold. Assuming that the weight threshold is 80%, when the repetition weight is the similarity between the current boundary information and the preset boundary information is less than 80%, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis model to redetermine the current behavior rule information. In the case where the repetition weight does not meet the preset threshold condition, the next cycle of video monitoring is performed, and the video image to be analyzed is updated in real time.

[0086] Through the above steps, when the scene image is fixed, the embodiment of the present application further determines whether the boundary information of the current cycle has changed. If it has changed, the current behavior rule information is re-determined. If there is no change, the behavior rule information is not repeatedly output, thereby improving the adaptive ability and saving resources.

[0087] In some embodiments, the rule analysis model includes a semantic understanding module and a multimodal analysis module, and step S204 includes:

[0088] Step S2041, in response to the behavior rule description instruction, the semantic understanding module outputs the semantic parsing result, and the multimodal analysis module outputs the multimodal analysis result.

[0089] Among them, the semantic understanding module can be a semantic understanding large model, and the multimodal analysis module can be a multimodal large model.

[0090] Step S2042: determining the behavior rule information corresponding to the video image to be analyzed according to the semantic parsing result and the multimodal analysis result.

[0091] The semantic parsing result is determined by understanding the semantic information of scene rule matching, and the multimodal analysis result is determined through the multimodal large model. Then, based on the semantic parsing result and the multimodal analysis result, the behavioral rule information corresponding to the video image to be analyzed is determined.

[0092] Through the above steps, the embodiment of the present application uses a semantic understanding big model to perform semantic analysis, converting text information into deep understanding, which can better understand and parse complex scene requirements, and uses a multimodal big model to perform multimodal analysis on real-time video streams, which can more comprehensively understand scene information, enhance the system's recognition and judgment capabilities, and further improve the degree of automation and accuracy of the analysis.

[0093] The embodiments of the present application are described and illustrated below through preferred embodiments.

[0094] Figure 4 is a flow chart of an image information analysis method according to a preferred embodiment of the present application, such as Figure 4 As shown, the image information analysis method includes the following steps:

[0095] Step S301, configuring a data acquisition channel of a video capture device, setting algorithm rules and description rule scenarios on the data acquisition channel to determine behavior rule description instructions.

[0096] Step S302, obtaining the collected video images to be analyzed and the behavior rule description instructions, and inputting the video images to be analyzed and the behavior rule description instructions into the rule analysis model including the semantic understanding module and the multimodal analysis module.

[0097] Step S303, in response to the behavior rule description instruction, the rule analysis model outputs rule direction information and rule coordinate information.

[0098] Step S304: sending the regular direction information and the regular coordinate information to the client device, and the client device displays the regular direction information and the regular coordinate information.

[0099] Step S305, in response to the rule information analysis instruction of the client device, the server device receives the rule direction information and the rule coordinate information and performs intelligent algorithm analysis on them, and determines the target boundary line and / or target boundary area in the video image to be analyzed.

[0100] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0101] This embodiment also provides an image information analysis device, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0102] Figure 5 is a structural block diagram of an image information analysis device according to an embodiment of the present application, such as Figure 5 As shown, the device includes an analysis object input module 10, a direction information determination module 20, a coordinate information determination module 30 and a rule information output module 40.

[0103] The analysis object input module 10 is used to obtain the video image to be analyzed and the behavior rule description instruction, and input the video image to be analyzed and the behavior rule description instruction into the rule analysis big model;

[0104] A direction information determination module 20, for responding to a behavior rule description instruction and determining rule direction information corresponding to the video image to be analyzed according to semantic description information contained in the behavior rule description instruction;

[0105] A coordinate information determination module 30, for extracting a feature vector of the video image to be analyzed in response to the behavior rule description instruction, and determining the rule coordinate information corresponding to the video image to be analyzed according to the feature vector;

[0106] The rule information output module 40 is used to determine the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, so that the rule analysis model outputs the behavior rule information.

[0107] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0108] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0109] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0110] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0111] Obtaining the video image to be analyzed and the behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into the rule analysis big model;

[0112] In response to the behavior rule description instruction, determining the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction;

[0113] In response to the behavior rule description instruction, extract the feature vector of the video image to be analyzed, and determine the rule coordinate information corresponding to the video image to be analyzed according to the feature vector;

[0114] According to the rule direction information and the rule coordinate information, the behavior rule information corresponding to the video image to be analyzed is determined, and the rule analysis model outputs the behavior rule information.

[0115] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0116] In addition, in combination with the image information analysis method in the above embodiment, the present application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the image information analysis methods in the above embodiment is implemented.

[0117] This embodiment also provides an image information analysis system, characterized in that it includes: a terminal device, a transmission device and a server device; wherein the terminal device is connected to the server device via the transmission device;

[0118] The terminal device is used to receive information sent by the server device and / or information input by the user, and to display information; the transmission device is used to transmit information; and the server device is used to execute any one of the image information analysis methods in the above embodiments.

[0119] The terminal device may include a client device, which may be used to receive and display the rule direction information and rule coordinate information sent by the server device, and may also be used to receive and send the rule information analysis instruction input by the user. The server device may be used to send the rule direction information and rule coordinate information to the server device.

[0120] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0122] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image information analysis method, characterized in that: The following steps are involved: Obtaining a video image to be analyzed and a behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into a rule analysis big model; In response to the behavior rule description instruction, determining the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction; In response to the behavior rule description instruction, extracting a feature vector of the video image to be analyzed, and determining rule coordinate information corresponding to the video image to be analyzed according to the feature vector; The behavior rule information corresponding to the video image to be analyzed is determined according to the rule direction information and the rule coordinate information, and the rule analysis model outputs the behavior rule information.

2. The image information analysis method according to claim 1, characterized in that: In determining the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, after the rule analysis model outputs the behavior rule information, the following further comprises: Sending the regular direction information and the regular coordinate information to a client device; In response to the rule information analysis instruction from the client device, the rule direction information and the rule coordinate information are sent to the server device.

3. The image information analysis method according to claim 2, characterized in that: After the rule information analysis instruction from the client device is sent to the server device, the method further includes: According to the regular coordinate information, the server device determines the target boundary line and / or target boundary area in the video image to be analyzed.

4. The image information analysis method according to any one of claims 1 to 3, characterized in that: The obtaining of the video image to be analyzed and the behavior rule description instruction, and inputting the video image to be analyzed and the behavior rule description instruction into the rule analysis big model comprises: Acquire a video image to be analyzed and a behavior rule description instruction, and determine a video capture device corresponding to the video image to be analyzed; In the case that the video shooting device is a non-fixed shooting device and the position of the non-fixed shooting device is updated, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis large model.

5. The image information analysis method according to claim 4, characterized in that: After obtaining the video image to be analyzed and the behavior rule description instruction, and determining the video shooting device corresponding to the video image to be analyzed, the method further includes: When the video capture device is a fixed capture device, or when the video capture device is a non-fixed capture device and the position of the non-fixed capture device is not updated, determining current boundary information in the video image to be analyzed; Calculating a repetition weight between the current boundary information and the preset boundary information; When the repetition weight meets a preset threshold condition, the video image to be analyzed and the behavior rule description instruction are input into the rule analysis model.

6. The image information analysis method according to claim 5, characterized in that: The rule analysis model includes a semantic understanding module and a multimodal analysis module. The behavior rule information output by the rule analysis model includes: In response to the behavior rule description instruction, the semantic understanding module outputs a semantic parsing result, and the multimodal analysis module outputs a multimodal analysis result; According to the semantic parsing result and the multimodal analysis result, the behavior rule information corresponding to the video image to be analyzed is determined.

7. An image information analysis device, characterized in that: It includes analysis object input module, direction information determination module, coordinate information determination module and rule information output module: The analysis object input module is used to obtain the video image to be analyzed and the behavior rule description instruction, and input the video image to be analyzed and the behavior rule description instruction into the rule analysis big model; The direction information determination module is used to respond to the behavior rule description instruction and determine the rule direction information corresponding to the video image to be analyzed according to the semantic description information contained in the behavior rule description instruction; The coordinate information determination module is used to extract the feature vector of the video image to be analyzed in response to the behavior rule description instruction, and determine the rule coordinate information corresponding to the video image to be analyzed according to the feature vector; The rule information output module is used to determine the behavior rule information corresponding to the video image to be analyzed according to the rule direction information and the rule coordinate information, so that the rule analysis model outputs the behavior rule information.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the image information analysis method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the image information analysis method according to any one of claims 1 to 6 when executed.

10. An image information analysis system, characterized in that: include: A terminal device, a transmission device and a server device; wherein the terminal device is connected to the server device via the transmission device; The terminal device is used to receive information sent by the server device and / or information input by the user, and display information; The transmission device is used to transmit information; The server device is used to execute the image information analysis method according to any one of claims 1 to 6.

Images