A video stream analysis and processing method, device, equipment and readable storage medium

Abstract

The present invention provides a method, device, equipment and readable storage medium for analyzing and processing a video stream, and relates to the technical field of video stream processing. The method comprises the following steps: collecting at least one video stream and arranging an analysis and processing model for the video stream; inputting the video stream into the correspondingly arranged analysis and processing model to obtain an inference result output by the analysis and processing model; sending the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; the message queue pushes the re-encoded inference video stream. During the push process of the present invention, the alarm information stored in the message queue can be pushed to a public message component for subscription by an upper-level business system, thereby forming a business closed loop, so that the camera can perform flexible video analysis on the video stream.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream processing, and in particular to a method, device, equipment and readable storage medium for analyzing and processing a video stream. Background Art

[0002] Currently, there are camera devices with artificial intelligence (AI) analysis functions. However, the AI ​​analysis capabilities of such camera devices are embedded in the built-in chips of the cameras, resulting in weak AI analysis computing power. As a result, the AI ​​analysis functions of such camera devices are relatively basic. In addition, due to the design of being embedded in the chip, the AI ​​capabilities of the camera devices cannot be flexibly replaced. For some relatively complex reasoning algorithms, the cameras cannot meet the requirements. Summary of the invention

[0003] The present invention provides a video stream analysis and processing method, device, equipment and readable storage medium, which are used to solve the defect in the prior art that the camera AI reasoning capability cannot be replaced and thus cannot meet application requirements, and realize flexible video analysis of the video stream.

[0004] The present invention provides a method for analyzing and processing a video stream, comprising the following steps:

[0005] Collect at least one video stream and compile an analysis and processing model for the video stream;

[0006] Input the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model; wherein the inference result includes alarm information, and when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of image information;

[0007] Sending the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; wherein the inference result is saved in a fixed format to the message queue;

[0008] The streaming service streams the re-encoded inference video stream stored in the message queue.

[0009] According to a video stream analysis and processing method provided by the present invention, the acquisition of at least one video stream, arranging an analysis and processing model for the video stream, and performing channel arrangement of the video stream specifically comprises the following steps:

[0010] Configuring an analysis and processing warehouse; wherein the analysis and processing warehouse stores the analysis and processing model and parameters corresponding to the analysis and processing model;

[0011] Collecting at least one video stream;

[0012] Based on the analysis and processing warehouse, the analysis and processing model is arranged for each video stream.

[0013] According to a video stream analysis and processing method provided by the present invention, the arrangement modes include one-to-one, one-to-many, many-to-one and many-to-many.

[0014] According to a video stream analysis and processing method provided by the present invention, the reasoning process of the analysis and processing model specifically includes the following steps:

[0015] Using the video stream as input data for inference, decoding and extracting frames of the video stream according to the parameters of the analysis and processing model, obtaining the image information of multiple consecutive frames, and storing the image information in a database;

[0016] The frame data is extracted and inferred from the database to generate the inference result.

[0017] According to a video stream analysis and processing method provided by the present invention, the database adopts a remote dictionary service.

[0018] According to a video stream analysis and processing method provided by the present invention, the sending of the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the frame number corresponding to the inference result to generate an inference video stream corresponding to the inference result specifically includes the following steps:

[0019] Sending at least one of the inference results corresponding to the video stream to the message queue;

[0020] The image information of the frames corresponding to each of the inference results is re-encoded to generate an inference video stream corresponding to the inference result.

[0021] The present invention also provides a video stream analysis and processing device, comprising:

[0022] A channel arrangement module, used for collecting at least one video stream and arranging an analysis and processing model for the video stream;

[0023] An inference module, used for inputting the video stream into the correspondingly arranged analysis and processing model to obtain an inference result output by the analysis and processing model; wherein the inference result includes an alarm message, and when the analysis and processing model infers the video stream, the video stream is decoded into image information of multiple consecutive frames;

[0024] An encoding module, used for sending the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result;

[0025] The alarm push module is used to push the re-encoded inference video stream stored in the message queue through a streaming service.

[0026] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned video stream analysis and processing methods are implemented.

[0027] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video stream analysis and processing methods are implemented.

[0028] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned methods for analyzing and processing video streams are implemented.

[0029] The video stream analysis and processing method, device, equipment and readable storage medium provided by the present invention can give the ordinary video stream acquisition side, such as camera equipment, AI reasoning and analysis capabilities by flexibly arranging the analysis and processing model with the video stream acquisition side channel, and decouple the AI ​​reasoning capability from the camera equipment. After the arrangement, the video stream is input into the corresponding arranged analysis and processing model to obtain the reasoning result output by the analysis and processing model. Moreover, when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of continuous image information. After that, the standard reasoning result data is returned to MQ and saved in a fixed format. Because the analyzed image information can be annotated during the analysis process, the analyzed annotated picture can be re-encoded into an inference video stream and pushed to the business system through MQ. The re-encoded inference video stream can display the reasoning results and the alarm information contained therein, so that the alarm information stored in MQ can be pushed to the public message component during the push process for the upper-level business system to subscribe to, forming a business closed loop. The camera device can therefore perform flexible video analysis on the video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flow chart of the video stream analysis and processing method provided by the present invention;

[0032] Figure 2 It is a specific flow chart of step S100 in the video stream analysis and processing method provided by the present invention;

[0033] Figure 3 It is a specific flow chart of step S300 in the video stream analysis and processing method provided by the present invention;

[0034] Figure 4 It is a structural schematic diagram of a video stream analysis and processing device provided by the present invention;

[0035] Figure 5 It is a specific structural diagram of the channel arrangement module in the video stream analysis and processing device provided by the present invention;

[0036] Figure 6 It is a specific structural diagram of the encoding module in the video stream analysis and processing device provided by the present invention;

[0037] Figure 7 It is a structural schematic diagram of the camera device provided by the present invention;

[0038] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Combine the following Figure 1 The video stream analysis and processing method of the present invention is described. The method is applied to the video stream acquisition side, such as a camera device, and the method comprises the following steps:

[0041] S100: Collect at least one video stream and compose an analysis and processing model for the received video stream.

[0042] What is performed in step S100 is the channel arrangement of the video stream. The channel arrangement setting is flexible, and a suitable analysis and processing model, i.e., a video analysis algorithm, is selected for the video content to be analyzed. Step S100 arranges the channels for the video stream, so that the method supports the analysis of a variety of video information, supports the direct analysis of the video information collected by the video stream acquisition side, such as the camera device, the analysis of the uploaded video file, the Real Time Streaming Protocol (RTSP) video stream analysis, etc.

[0043] Since there can be multiple video stream acquisition sides connected and multiple scene types that need AI analysis can also be performed, the channel arrangement methods in this method include one-to-one, one-to-many, many-to-one and many-to-many. One-to-one means that one video stream acquisition side is bound to and arranges one analysis and processing model, so that the video stream acquisition side has AI analysis capabilities in one situation. One-to-many means that one video stream acquisition side is bound to and arranges multiple analysis and processing models, so that the video stream acquisition side has AI analysis capabilities in multiple situations at the same time. Many-to-one means that multiple video stream acquisition sides are bound to and arrange the same analysis and processing model, and these video stream acquisition sides have AI analysis capabilities in the same situation. Many-to-many means that multiple video stream acquisition sides are bound to and arrange multiple analysis and processing models, and each video stream acquisition side has AI analysis capabilities in multiple situations at the same time. Moreover, the AI ​​analysis capabilities of each video stream acquisition side do not need to be the same, and specific AI analysis capabilities can be arranged according to actual application scenarios.

[0044] Each bound video stream acquisition side (video stream channel) can be used as an analysis channel, so that efficient combination can be performed in the subsequent analysis of the video stream. By establishing different analysis channels for the video streams that need to be analyzed, one camera can be used in multiple analysis scenarios. At the same time, it is convenient to send the AI ​​inference results of the video stream to the public message component for subscription by the upper-level business system.

[0045] It should be noted that when a new video stream acquisition side (video stream channel) is added, the type of the video stream channel can be set. The video stream acquisition side can use video files and real-time video streams as video sources.

[0046] S200, input the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model, wherein the inference result includes warning information, and when the analysis and processing model infers the video stream, it decodes the video stream into multiple frames of image information, i.e., pictures.

[0047] Specifically, the reasoning process of analyzing and processing the model in step S200 specifically includes the following steps:

[0048] S210, using the video stream as input data for reasoning, decoding the frame-by-frame video stream according to the parameters of the analysis and processing model, obtaining image information of multiple consecutive frames, and storing the image information in a database.

[0049] S220, extracting frame data from the database and performing AI inference to generate inference results.

[0050] In this embodiment, the image information in step S200 is stored in a database. Preferably, the database can use a remote dictionary server (Remote Dictionary Server, Redis). Similarly, the image information is obtained from Redis and analyzed and reasoned to obtain the reasoning result to determine whether there is data that needs to be alerted, such as analyzing whether there is a worker not wearing a helmet through the algorithm corresponding to the helmet scene analysis processing model, and analyzing whether there is a worker not wearing a mask through the algorithm corresponding to the mask scene analysis processing model.

[0051] S300, sending the inference result corresponding to the video stream to the message queue (MQ), and re-encoding the image information corresponding to the number of frames of the inference result. Specifically, re-encoding the image information corresponding to the inference result obtained by AI inference, i.e., the picture, frame by frame, to generate an inference video stream corresponding to the inference result. The inference video stream after re-encoding and annotating in this way can clearly show the result of AI inference.

[0052] In this embodiment, the inference result is saved to MQ in a fixed format to facilitate the subsequent parsing and pushing of the alarm information. Specifically, the fixed format in this method is the Json data format, for example:

[0053]

[0054]

[0055] S400, the streaming service pushes the re-encoded inference video stream stored in MQ. For example, by pushing the encoded inference video stream to the public message component, the alarm information stored in MQ is sent to the public message component for subscription by the upper-level business system, thereby forming a business closed loop.

[0056] The video stream analysis and processing method of the present invention can give the ordinary video stream acquisition side, such as the camera device, AI reasoning and analysis capabilities by flexibly arranging the analysis and processing model with the video stream acquisition side channel, and decouple the AI ​​reasoning capability from the camera device. After the arrangement, the video stream is input into the corresponding arranged analysis and processing model to obtain the reasoning result output by the analysis and processing model. Moreover, when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of continuous image information. After that, the standard reasoning result data is returned to MQ and saved in a fixed format. Because the analyzed image information can be annotated during the analysis process, the analyzed annotated picture can be re-encoded into an inference video stream and pushed to the business system through MQ. The re-encoded inference video stream can display the reasoning results and the alarm information contained therein, so that the alarm information stored in MQ can be pushed to the public message component during the push process for the upper-level business system to subscribe to, forming a business closed loop. The camera device can therefore perform flexible video analysis on the video stream.

[0057] Combine the following Figure 2 The video stream analysis and processing method of the present invention is described, and step S100 specifically includes the following steps:

[0058] S110, configure the analysis and processing warehouse. The analysis and processing warehouse stores the analysis and processing model and the parameters corresponding to the analysis and processing model, including the associated AI algorithm, the setting of the image saving type, the setting of the number of frames per second, the minimum side length of the image, the expiration time, the alarm label, the message push type, etc. When setting the corresponding analysis parameters of the video stream in step S110, multiple parameter combination settings are supported.

[0059] The configuration analysis and processing warehouse can be specifically as follows: first, the analysis and processing model is trained through a large number of sample sets, and finally an available analysis and processing model is formed. It should be noted that this method also supports access to available analysis and processing models of a third party, as long as the analysis and processing model is processed by step S200 and step S300, and the data format obtained is a fixed format in this method. Afterwards, the available analysis and processing model is published and put on the shelf, and the corresponding parameters are configured for the published analysis and processing model. The parameters of the analysis and processing model after publication can be manually set, for example, the face recognition algorithm can be configured with face library information, the regional intrusion algorithm can be configured with the coordinate information of the region, etc. Different parameters can be configured for different analysis and processing models. Afterwards, the parameters of the analysis and processing model can be updated in real time when the version is upgraded.

[0060] S120: Collect at least one video stream.

[0061] S130: Based on the analysis and processing warehouse, compose an analysis and processing model for each video stream.

[0062] The channel arrangement modes in the method include one-to-one, one-to-many, many-to-one and many-to-many. One-to-one means that one video stream acquisition side is bound to and arranges one analysis and processing model, so that the video stream acquisition side has AI analysis capability in one situation. One-to-many means that one video stream acquisition side is bound to and arranges multiple analysis and processing models, so that the video stream acquisition side has AI analysis capability in multiple situations at the same time. Many-to-one means that multiple video stream acquisition sides are bound to and arrange the same analysis and processing model, and these video stream acquisition sides have AI analysis capability in the same situation. Many-to-many means that multiple video stream acquisition sides are bound to and arrange multiple analysis and processing models, and each video stream acquisition side has AI analysis capability in multiple situations at the same time. Moreover, the AI ​​analysis capability of each video stream acquisition side does not need to be the same. Specific AI analysis capability can be arranged according to the actual application scenario, and one video stream can be analyzed from multiple angles.

[0063] Combine the following Figure 3 The video stream analysis and processing method of the present invention is described. Since each video stream may have AI analysis capabilities in multiple situations at the same time, after the AI ​​reasoning in step S200, multiple reasoning results may be obtained. Therefore, step S300 specifically includes the following steps:

[0064] S310, sending at least one inference result corresponding to the video stream to MQ.

[0065] S310, re-encode the image information of the frames corresponding to each inference result to generate an inference video stream corresponding to the inference result, wherein each inference result corresponds to an inference video stream.

[0066] The video stream analysis and processing device provided by the present invention is described below. The video stream analysis and processing device described below and the video stream analysis and processing method described above can refer to each other.

[0067] Combine the following Figure 4 The video stream analysis and processing device of the present invention is described. The device is used in a video stream acquisition side, such as a camera device, and the device includes:

[0068] The channel arrangement module A100 is used to collect at least one video stream and arrange an analysis and processing model for the accessed video stream.

[0069] The channel arrangement module A100 performs channel arrangement of the video stream. The channel arrangement setting is flexible, and a suitable analysis and processing model, i.e., a video analysis algorithm, is selected for the video content to be analyzed. The channel arrangement module A100 performs channel arrangement for the video stream, so that the device supports the analysis of various video information, and supports the direct analysis of video information collected by the video stream acquisition side, such as the analysis of uploaded video files, and the analysis of RTSP video streams.

[0070] Since there can be multiple video stream acquisition sides connected and multiple scene types that need AI analysis can also be performed, the channel arrangement methods in the device include one-to-one, one-to-many, many-to-one and many-to-many. One-to-one means that one video stream acquisition side is bound to and arranged with one analysis and processing model, so that the video stream acquisition side has AI analysis capabilities in one situation. One-to-many means that one video stream acquisition side is bound to and arranged with multiple analysis and processing models, so that the video stream acquisition side has AI analysis capabilities in multiple situations at the same time. Many-to-one means that multiple video stream acquisition sides are bound to and arranged with the same analysis and processing model, and these video stream acquisition sides have AI analysis capabilities in the same situation. Many-to-many means that multiple video stream acquisition sides are bound to and arranged with multiple analysis and processing models, and each video stream acquisition side has AI analysis capabilities in multiple situations at the same time. Moreover, the AI ​​analysis capabilities of each video stream acquisition side do not need to be the same, and specific AI analysis capabilities can be arranged according to actual application scenarios.

[0071] Each bound video stream acquisition side (video stream channel) can be used as an analysis channel, so that efficient combination can be performed in the subsequent analysis of the video stream. By establishing different analysis channels for the video streams that need to be analyzed, one camera can be used in multiple analysis scenarios. At the same time, it is convenient to send the AI ​​inference results of the video stream to the public message component for subscription by the upper-level business system.

[0072] It should be noted that when a new video stream acquisition side (video stream channel) is added, the type of the video stream channel can be set. The video stream acquisition side can use video files and real-time video streams as video sources.

[0073] The inference module A200 is used to input the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model. The inference result includes alarm information. When the analysis and processing model infers the video stream, it decodes the video stream into continuous multiple frames of image information, i.e., pictures.

[0074] Specifically, the reasoning process of the analysis and processing model in the reasoning module A200 specifically includes the following steps:

[0075] The decoding and frame extraction unit A210 is used to use the video stream as input data for inference, decode the frame-extracted video stream according to the parameters of the analysis and processing model, obtain image information of multiple consecutive frames, and save the image information in a database.

[0076] The inference unit A220 is used to extract and perform AI inference on frame data from the database to generate inference results.

[0077] In this embodiment, the image information in the reasoning module A200 is stored in a database. Preferably, the database can use Redis. Similarly, the image information is obtained from Redis and analyzed and reasoned to generate reasoning results to determine whether there is data that needs to be alerted, such as analyzing whether there is a worker not wearing a helmet through the algorithm corresponding to the helmet scene analysis processing model, and analyzing whether there is a worker not wearing a mask through the algorithm corresponding to the mask scene analysis processing model.

[0078] The encoding module A300 sends the inference result to MQ and re-encodes the video data corresponding to the inference result. Specifically, the image information and pictures corresponding to the frame number of the inference result obtained by AI inference are re-encoded frame by frame to generate an inference video stream corresponding to the inference result. The inference video stream after re-encoding and annotating in this way can clearly show the result of AI inference.

[0079] In this embodiment, the inference result is saved to MQ in a fixed format to facilitate the subsequent parsing and pushing of the alarm information. Specifically, the fixed format in the device is the Json data format.

[0080] The alarm push module A400 is used to push the re-encoded inference video stream saved in MQ through the streaming service. For example, by pushing the encoded inference video stream to the public message component, the alarm information stored in MQ is sent to the public message component for subscription by the upper-level business system, thereby forming a business closed loop.

[0081] The video stream analysis and processing device of the present invention can give the ordinary video stream acquisition side, such as the camera device, AI reasoning and analysis capabilities by flexibly arranging the analysis and processing model with the video stream acquisition side channel, and decouple the AI ​​reasoning capability from the camera device. After the arrangement, the video stream is input into the corresponding arranged analysis and processing model to obtain the reasoning result output by the analysis and processing model. Moreover, when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of continuous image information. Thereafter, the standard reasoning result data is returned to MQ and saved in a fixed format. Because the analyzed image information can be annotated during the analysis process, the analyzed annotated picture can be re-encoded into an inference video stream and pushed to the business system through MQ. The re-encoded inference video stream can display the reasoning results and the alarm information contained therein, so that the alarm information stored in MQ can be pushed to the public message component during the push process for the upper-level business system to subscribe to, thereby forming a business closed loop. The camera device can therefore perform flexible video analysis on the video stream.

[0082] Combine the following Figure 5 The video stream analysis and processing device of the present invention is described, and the channel arrangement module A100 specifically includes:

[0083] Configuration unit A110 is used to configure the analysis and processing warehouse. The analysis and processing warehouse stores the analysis and processing model and the parameters corresponding to the analysis and processing model, including the associated AI algorithm, the setting of the image saving type, the setting of the number of frames per second, the minimum side length of the image, the expiration time, the alarm label, the message push type, etc. When setting the corresponding analysis parameters of the video stream in step S110, multiple parameter combination settings are supported.

[0084] The configuration analysis and processing warehouse can be specifically as follows: first, a large number of sample sets are used to train the analysis and processing model, and finally a usable analysis and processing model is formed. It should be noted that the device also supports access to available analysis and processing models of a third party, as long as the analysis and processing model is processed by the decoding frame extraction module A200 and the reasoning module A200, the obtained data format is a fixed format in the device. Afterwards, the available analysis and processing model is published and put on the shelf, and the corresponding parameters are configured for the published analysis and processing model. The parameters of the analysis and processing model after publication can be manually set, for example, the face recognition algorithm can be configured with face library information, the regional intrusion algorithm can be configured with the coordinate information of the region, etc. Different parameters can be configured for different analysis and processing models. Afterwards, the parameters of the analysis and processing model can be updated in real time when the version is upgraded.

[0085] The acquisition unit A120 is used to acquire at least one video stream.

[0086] The channel arrangement unit A130 is used to arrange the analysis and processing model for each video stream based on the analysis and processing warehouse.

[0087] The channel arrangement modes in the device include one-to-one, one-to-many, many-to-one and many-to-many. One-to-one means that one video stream acquisition side is bound to and arranged with one analysis and processing model, so that the video stream acquisition side has AI analysis capability in one situation. One-to-many means that one video stream acquisition side is bound to and arranged with multiple analysis and processing models, so that the video stream acquisition side has AI analysis capability in multiple situations at the same time. Many-to-one means that multiple video stream acquisition sides are bound to and arranged with the same analysis and processing model, and these video stream acquisition sides have AI analysis capability in the same situation. Many-to-many means that multiple video stream acquisition sides are bound to and arranged with multiple analysis and processing models, and each video stream acquisition side has AI analysis capability in multiple situations at the same time. Moreover, the AI ​​analysis capability of each video stream acquisition side does not need to be the same. Specific AI analysis capability can be arranged according to the actual application scenario, and one video stream can be analyzed from multiple angles.

[0088] Combine the following Figure 6The video stream analysis and processing device of the present invention is described. Since each video stream may have AI analysis capabilities in multiple situations at the same time, after AI reasoning by the reasoning module A200, multiple reasoning results may be obtained. Therefore, the encoding module A300 specifically includes:

[0089] The return unit A310 is used to send at least one inference result corresponding to the video stream to MQ.

[0090] The encoding unit A310 is used to re-encode the image information of the frames corresponding to each inference result to generate an inference video stream corresponding to the inference result, and each inference result corresponds to an inference video stream.

[0091] Combine the following Figure 7 The camera device of the present invention is described, and the camera device comprises:

[0092] The channel arrangement module B100 is used to collect at least one video stream and arrange an analysis and processing model for the accessed video stream.

[0093] The channel arrangement module B100 performs channel arrangement of video streams. The channel arrangement setting is flexible, and a suitable analysis and processing model, i.e., a video analysis algorithm, is selected for the video content to be analyzed. The channel arrangement module B100 arranges channels for video streams, so that the camera device supports the analysis of various video information, and supports the video stream acquisition side, such as direct analysis of video information collected by the camera device, analysis of uploaded video files, and analysis of RTSP video streams.

[0094] Since there can be multiple camera devices connected and multiple scene types that need AI analysis, the channel arrangement methods include one-to-one, one-to-many, many-to-one and many-to-many. One-to-one means that one camera device is bound to and arranged with one analysis and processing model, so that the camera device has AI analysis capability in one situation. One-to-many means that one camera device is bound to and arranged with multiple analysis and processing models, so that the camera device has AI analysis capability in multiple situations at the same time. Many-to-one means that multiple camera devices are bound to and arranged with the same analysis and processing model, and these camera devices have AI analysis capability in the same situation. Many-to-many means that multiple camera devices are bound to and arranged with multiple analysis and processing models, and each camera device has AI analysis capability in multiple situations at the same time. Moreover, the AI ​​analysis capability of each camera device does not need to be the same, and specific AI analysis capability arrangement can be performed according to the actual application scenario.

[0095] Each bound camera device (video stream channel) can be used as an analysis channel, so that efficient combination can be performed in the subsequent analysis of the video stream. By establishing different analysis channels for the video streams that need to be analyzed, one camera can be used in multiple analysis scenarios. At the same time, it is convenient to send the AI ​​inference results of the video stream to the public message component for subscription by the upper-level business system.

[0096] It should be noted that when adding a new camera device (video stream channel), the type of the video stream channel can be set, and the video stream acquisition side can use video files and real-time video streams as video sources.

[0097] The inference module B200 is used to input the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model. The inference result includes alarm information. When the analysis and processing model infers the video stream, it decodes the video stream into continuous multiple frames of image information, i.e., pictures.

[0098] Specifically, the reasoning process of the analysis and processing model in the reasoning module B200 specifically includes the following steps:

[0099] The decoding and frame extraction unit B210 is used to use the video stream as input data for inference, decode the frame-extracted video stream according to the parameters of the analysis and processing model, obtain image information of multiple consecutive frames, and save the image information in a database.

[0100] The inference unit B220 is used to extract and perform AI inference on frame data from the database to generate inference results.

[0101] In this embodiment, the image information in the reasoning module B200 is stored in a database. Preferably, the database can use Redis. Similarly, the image information is obtained from Redis and analyzed and reasoned to generate reasoning results to determine whether there is data that needs to be alerted, such as analyzing whether there is a worker not wearing a helmet through the algorithm corresponding to the helmet scene analysis processing model, and analyzing whether there is a worker not wearing a mask through the algorithm corresponding to the mask scene analysis processing model.

[0102] The encoding module B300 sends the inference result to MQ and re-encodes the image information corresponding to the number of frames of the inference result. Specifically, the image information corresponding to the inference result obtained by AI inference, i.e., the picture, is re-encoded frame by frame to generate an inference video stream corresponding to the inference result. The inference video stream after re-encoding and annotating in this way can clearly show the result of AI inference.

[0103] In this embodiment, the inference result is saved to MQ in a fixed format to facilitate the subsequent parsing and pushing of the alarm information. Specifically, the fixed format in the camera device is the Json data format.

[0104] The alarm push module B400 is used to push the re-encoded inference video stream saved in MQ through the streaming service. For example, by pushing the encoded inference video stream to the public message component, the alarm information stored in MQ is sent to the public message component for subscription by the upper-level business system, thereby forming a business closed loop.

[0105] The camera device of the present invention can give ordinary camera devices AI reasoning and analysis capabilities by flexibly arranging the analysis and processing model with the camera device, and decouple the AI ​​reasoning capability from the camera device. After arrangement, the video stream is input into the correspondingly arranged analysis and processing model to obtain the reasoning result output by the analysis and processing model, and when the analysis and processing model infers the video stream, the video stream is decoded into continuous multiple frames of image information, and then the standard reasoning result data is returned to MQ and saved in a fixed format. Because the analyzed image information can be annotated during the analysis process, the analyzed annotated picture can be re-encoded into a reasoning video stream and pushed to the business system through MQ. The re-encoded reasoning video stream can display the reasoning result and the alarm information contained therein, so that the alarm information stored in MQ can be pushed to the public message component during the push process for the upper-level business system to subscribe, forming a business closed loop, and the camera device can therefore perform flexible video analysis on the video stream. .

[0106] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the video stream analysis and processing method, which includes the following steps:

[0107] S100, collecting at least one video stream, and composing an analysis and processing model for the video stream;

[0108] S200, inputting the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model; wherein the inference result includes alarm information, and when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of image information;

[0109] S300, sending the inference result corresponding to the video stream to MQ, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; wherein the inference result is saved in the MQ in a fixed format;

[0110] S400: The streaming service pushes the re-encoded inference video stream stored in the MQ.

[0111] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0112] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can perform the video stream analysis and processing method provided by the above methods, the method comprising the following steps:

[0113] S100, collecting at least one video stream, and composing an analysis and processing model for the video stream;

[0114] S200, inputting the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model; wherein the inference result includes alarm information, and when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of image information;

[0115] S300, sending the inference result corresponding to the video stream to MQ, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; wherein the inference result is saved in the MQ in a fixed format;

[0116] S400: The streaming service pushes the re-encoded inference video stream stored in the MQ.

[0117] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned video stream analysis and processing method, the method comprising the following steps:

[0118] S100, collecting at least one video stream, and composing an analysis and processing model for the video stream;

[0119] S200, inputting the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model; wherein the inference result includes alarm information, and when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of image information;

[0120] S300, sending the inference result corresponding to the video stream to MQ, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; wherein the inference result is saved in the MQ in a fixed format;

[0121] S400: The streaming service pushes the re-encoded inference video stream stored in the MQ.

[0122] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for analyzing and processing a video stream, characterized in that: The following steps are involved: Collect at least one video stream and compile an analysis and processing model for the video stream; Input the video stream into the correspondingly arranged analysis and processing model to obtain the inference result output by the analysis and processing model; wherein the inference result includes alarm information, and when the analysis and processing model infers the video stream, the video stream is decoded into multiple frames of image information; Sending the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; wherein the inference result is saved in a fixed format to the message queue; The streaming service streams the re-encoded inference video stream stored in the message queue.

2. The video stream analysis and processing method according to claim 1, characterized in that: The step of collecting at least one video stream and arranging an analysis and processing model for the video stream specifically includes the following steps: Configuring an analysis and processing warehouse; wherein the analysis and processing warehouse stores the analysis and processing model and parameters corresponding to the analysis and processing model; Collecting at least one video stream; Based on the analysis and processing warehouse, the analysis and processing model is arranged for each video stream.

3. The video stream analysis and processing method according to claim 2, characterized in that: The arrangement modes include one-to-one, one-to-many, many-to-one and many-to-many.

4. The video stream analysis and processing method according to claim 1, characterized in that: The reasoning process of the analysis and processing model specifically includes the following steps: Using the video stream as input data for inference, decoding and extracting frames of the video stream according to the parameters of the analysis and processing model, obtaining the image information of multiple consecutive frames, and storing the image information in a database; The frame data is extracted and inferred from the database to generate the inference result.

5. The video stream analysis and processing method according to claim 4, characterized in that: The database uses a remote dictionary service.

6. The video stream analysis and processing method according to claim 3, characterized in that: The sending of the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the frames corresponding to the inference result to generate an inference video stream corresponding to the inference result specifically includes the following steps: Sending at least one of the inference results corresponding to the video stream to the message queue; The image information of the frames corresponding to each of the inference results is re-encoded to generate an inference video stream corresponding to the inference result.

7. A video stream analysis and processing device, characterized in that: include: A channel arrangement module, used for collecting at least one video stream and arranging an analysis and processing model for the video stream; An inference module, used for inputting the video stream into the correspondingly arranged analysis and processing model to obtain an inference result output by the analysis and processing model; wherein the inference result includes an alarm message, and when the analysis and processing model infers the video stream, the video stream is decoded into image information of multiple consecutive frames; An encoding module, used for sending the inference result corresponding to the video stream to a message queue, and re-encoding the image information of the number of frames corresponding to the inference result to generate an inference video stream corresponding to the inference result; The alarm push module is used to push the re-encoded inference video stream stored in the message queue through a streaming service.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the video stream analysis and processing method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video stream analysis and processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the video stream analysis and processing method according to any one of claims 1 to 6 are implemented.

Images