Video monitoring method, video monitoring system and terminal device
By transmitting the original image to the artificial intelligence module for analysis in the image monitoring system and displaying the analysis results on the terminal device, the problem that traditional devices cannot analyze and mark all types of objects in remote scenes is solved, achieving no delay image monitoring effect.
Patent Information
- Application Number
- CN202411103953.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional image monitoring devices cannot mark the analysis results of all types of objects that need to be judged in the remote scene, resulting in the inability to provide good image monitoring effects, especially in scenarios where original images and remote secondary processing are required.
The original image is obtained through the camera module and provided to the artificial intelligence module and server. The artificial intelligence module analyzes the image and generates analysis results. The server provides the original image and analysis results to the terminal device. The terminal device displays the original image, analysis results or superimposed images, where The superimposed image is part of the analysis result and superimposed on the original image.
It realizes displaying original images, analysis results or superimposed images without delay, provides good image monitoring effects, and meets the statistical needs of analysis results of specific types of objects.
Smart Images

Figure CN119071435A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a display technology, and more particularly to an image monitoring method, an image monitoring system and a terminal device. Background Art
[0002] Generally speaking, conventional video surveillance equipment used for video surveillance only analyzes the video and transmits the analysis results back to the computing host, which is quite inconvenient for certain scenarios that require original video and / or require remote secondary processing. In particular, for scenarios that require remote secondary processing, if the backend needs to collect statistics on the analysis results of a specific type of object, conventional video surveillance equipment cannot mark the analysis results of all types of objects that need to be judged in the video in advance, and therefore cannot provide good video surveillance effects. Summary of the invention
[0003] The present invention provides an image monitoring method, an image monitoring system and a terminal device, which can achieve good image monitoring effect.
[0004] The image monitoring method of the present invention includes the following steps: obtaining an original image through a camera module, and providing the original image to an artificial intelligence module and a server; analyzing the original image through the artificial intelligence module to generate an analysis result, and providing the analysis result to the server; providing the original image and the analysis result to a terminal device through the server; and displaying the original image, the analysis result or the superimposed image through the terminal device, wherein the superimposed image is a result of superimposing a part of the analysis result on the original image.
[0005] The image monitoring system of the present invention includes a terminal device, a server, an artificial intelligence module and a camera module. The server is coupled to the terminal device. The artificial intelligence module is coupled to the server. The camera module is coupled to the artificial intelligence module and the server. The camera module obtains an original image and provides the original image to the artificial intelligence module and the server. The artificial intelligence module analyzes the original image to generate an analysis result and provides the analysis result to the server. The server provides the original image and the analysis result to the terminal device. The terminal device displays the original image, the analysis result or the superimposed image. The superimposed image is a result in which a part of the analysis result is superimposed on the original image.
[0006] The terminal device of the present invention includes a display unit, a communication interface and a processing unit. The communication interface is coupled to the server and is used to receive the original image and the analysis result. The processing unit is coupled to the display unit and the communication interface and is used to display the original image, the analysis result or the superimposed image through the display unit. The superimposed image is a result of superimposing a part of the analysis result on the original image. The original image is provided via the camera module, and the analysis result is provided via the artificial intelligence module according to the original image.
[0007] Based on the above, the image monitoring method, image monitoring system and terminal device of the present invention can effectively display the original image, analysis result or superimposed image. The superimposed image is the result of superimposing a part of the analysis result on the original image.
[0008] In order to make the above features and advantages of the present invention more obvious and easy to understand, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 FIG. 1 is a schematic diagram of an image monitoring system according to an embodiment of the present invention.
[0010] Figure 2 is a schematic diagram of a terminal device according to an embodiment of the present invention.
[0011] Figure 3 It is a flow chart of an image monitoring method according to an embodiment of the present invention.
[0012] Figure 4 It is a schematic diagram of an original image, an analysis result, and a superimposed image according to an embodiment of the present invention.
[0013] Figure 5 It is a schematic diagram of the data format of tag information according to an embodiment of the present invention.
[0014] The reference numerals in the accompanying drawings are explained as follows: 100: image monitoring system; 101: original image; 102: image number; 103: analysis result; 104: image number; 105: camera module setting parameters; 106: artificial intelligence module setting parameters; 107: superimposed image; 110: edge device; 111: camera module; 112: artificial intelligence module; 120: server; 130: terminal device; 131: processing unit; 132: communication interface; 133: display unit; 134: input interface; 500: tag data; 501: confidence value; 502: font size; 503: tag number; 504: tag type; 505: X-axis coordinate; 506: Y-axis coordinate; 507: box width; 508: box height; B1~B7: bounding box; S310~S340: steps. DETAILED DESCRIPTION
[0015] In order to make the content of the present invention more clearly understood, the following embodiments are specifically cited as examples by which the present invention can be truly implemented. In addition, wherever possible, the same reference numerals are used in the drawings and embodiments to represent the same or similar components.
[0016] Figure 1 FIG. 1 is a schematic diagram of an image monitoring system according to an embodiment of the present invention. Figure 1, the image monitoring system 100 includes a camera module 111, an artificial intelligence (AI) module 112, a server 120, and a terminal device 130. In the present embodiment, the camera module 111 and the artificial intelligence module 112 can be set in the same edge device 110, but the present invention is not limited to this. In one embodiment, the camera module 111 and the artificial intelligence module 112 can also be set in different devices. In the present embodiment, the camera module 111 is coupled to the artificial intelligence module 112 and the server 120. The artificial intelligence module 112 is coupled to the server 120. The server 120 is coupled to the terminal device 130.
[0017] In this embodiment, the edge device 110 may be set, for example, in a remote image monitoring scene such as a manufacturing production line, a traffic intersection or a specific monitoring environment, and may realize edge computing functions. The edge device 110 may also be equipped with a processor and a memory to analyze the real-time image data generated by the camera module 111 through shooting by executing the artificial intelligence module 112, and generate analysis results. In this embodiment, the edge device 110 may be connected to the server 120, for example, by a wired or wireless communication method. The wired communication method may be, for example, a cable. The wireless communication method may be, for example, implemented through Wi-Fi, Bluetooth or other wireless communication interfaces. In this embodiment, the server 120 may be a cloud server or a web server, but the present invention is not limited thereto. In one embodiment, the server 120 may also be a ground device and be arranged in the same scene as the edge device 110.
[0018] In this embodiment, the terminal device 130 may be, for example, a smart phone, a personal computer (PC), a notebook, a tablet, or a related electronic device having a display function and supporting a web browser. The terminal device 130 may be connected to the server 120 via a wired or wireless communication method to receive the real-time monitoring image data and analysis results without delay provided by the server 120. In this embodiment, the terminal device 130 may, for example, select to display the real-time monitoring image, the real-time analysis result, or an image combined with a portion of the analysis result on the real-time monitoring image according to a preset display setting or a user setting.
[0019] The processor described in the embodiments of the present invention may include, for example, a central processing unit (CPU), or other programmable general or special microprocessors, digital signal processors (DSP), image processors (IPU), graphics processors (GPU), programmable controllers, application specific integrated circuits (ASIC), programmable logic devices (PLD), other similar processing devices or combinations of these devices. The memory described in the embodiments of the present invention may include, for example, dynamic random access memory (DRAM), flash memory or non-volatile random access memory (NVRAM).
[0020] Figure 2 is a schematic diagram of a terminal device according to an embodiment of the present invention. Figure 1 as well as Figure 2 In this embodiment, the terminal device 130 includes a processing unit 131, a communication interface 132, a display unit 133, and an input interface 134. The processing unit 131 is coupled to the communication interface 132, the display unit 133, and the input interface 134. In this embodiment, the terminal device 130 can receive the real-time monitoring image data and the analysis results provided by the server 120 through the communication interface 132, and display the corresponding monitoring image through the display unit 133. In one embodiment, the terminal device 130 can also receive the setting data for setting at least one of the camera module 111 and the artificial intelligence module 112 through the input interface 134, and send it to the server 120 through the communication interface 132, and then provide it to at least one of the camera module 111 and the artificial intelligence module 112 through the server 120.
[0021] In the present embodiment, the processing unit 131 includes a processor and a memory. In the present embodiment, the communication interface 132 may be a wired or wireless communication interface. In the present embodiment, the display unit 133 may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display or other types of displays, but the present invention is not limited thereto. In the present embodiment, the input interface 134 may be, for example, an input device such as a touch panel, a mouse or a keyboard that is integrated or not integrated with the display unit 133.
[0022] Figure 3 FIG. 1 is a flow chart of an image monitoring method according to an embodiment of the present invention. Figures 1 to 3 , the image monitoring system 100 may perform the following steps S310 to S340. In step S310, the camera module 111 may obtain the original image 101, and provide the original image 101 to the artificial intelligence module 112 and the server 120. In step S320, the artificial intelligence module 112 may analyze the original image 101 to generate an analysis result 103, and provide the analysis result 103 to the server 120. In this embodiment, the artificial intelligence module 112 may, for example, include a classification model implemented by one or more neural network models (NN), and is trained to be able to recognize specific types of objects in the image (such as human images, facial images, or object appearances), and mark specific objects to generate an analysis result 103 having at least one marking information.
[0023] In step S330, the server 120 may provide the original image 101 and the analysis result 103 to the terminal device 130. In step S340, the terminal device 130 may display the original image 101, the analysis result 103, or the superimposed image. In this embodiment, the superimposed image may be a result in which a portion of the analysis result 103 is superimposed on the original image 101. In other words, the terminal device 130 may display the original surveillance image, the image of the analysis result, or the surveillance image superimposed with a portion of the tag information. The tag information may include, for example, a tag type, a tag number, a confidence value, and tag position information, but the present invention is not limited thereto.
[0024] In this embodiment, the camera module 111 may number each frame of the original image 101 to synchronously provide the original image 101 and the image number 102 corresponding to each frame of the original image 101 to the artificial intelligence module 112 and the server 120. The artificial intelligence module 112 may identify a specific type of object in each frame of the original image 101 to generate an analysis result 103 corresponding to each frame of the original image 101. The artificial intelligence module 112 may synchronously provide the analysis result 103 and the image number 104 corresponding to each frame of the analysis result 103 to the server 120.
[0025] In this embodiment, the server 120 may synchronize the original image 101 and the analysis result 103 according to the image number 102 corresponding to the original image 101 and the image number 104 corresponding to the analysis result 103. In this regard, since the artificial intelligence module 112 may cause a delay in the process of analyzing the original image 101, the original image 101 of the real-time monitoring result and the analysis result 103 generated by the artificial intelligence module 112 are transmitted separately in this embodiment, and matched with the corresponding image numbers, so as to perform frame synchronization processing on the server 120, so that the server 120 can provide the synchronized original image 101, the analysis result 103, the image number 102 corresponding to the original image 101, and the image number 104 corresponding to the analysis result 103 to the terminal device 130. In this way, the terminal device 130 can display the original image 101, the analysis result 103 or the superimposed image (which may have a delay) without delay according to the user operation, display requirements or preset display settings, wherein the superimposed image may be the result of superimposing a part of the marking information of the analysis result 103 on the original image 101. Therefore, the video monitoring system 100 and the terminal device 130 can achieve good video monitoring effects.
[0026] In this embodiment, the terminal device 130 may also transmit back at least one of the camera module setting parameters 105 and the artificial intelligence module setting parameters 106 to the server 120. The server 120 may output at least one of the camera module setting parameters 105 and the artificial intelligence module setting parameters 106 to at least one of the camera module 111 and the artificial intelligence module 112. In this embodiment, the camera module setting parameters 105 may include, for example, parameters such as scene mode parameters, white balance parameters, focus, frame rate (FPS) or resolution, and the present invention is not limited thereto. In this embodiment, the artificial intelligence module setting parameters 106 may include instructions for the user to control and set the artificial intelligence module 112. The artificial intelligence module 112 may, for example, provide a parameter list (for example, including weight parameter settings or accelerated operation parameter settings) available for adjustment to the terminal device 130, so that the user can set it through the input interface 134 of the terminal device 130 and generate corresponding setting parameters.
[0027] In one embodiment, the user can also annotate the object image in the analysis result 103 displayed by the display unit 133 through the input interface 134 of the terminal device 130. For example, the user can annotate the corresponding detailed object type for the object determined in the analysis result 103. In other words, the artificial intelligence module setting parameter 106 can also include the annotation information corresponding to the marked result in the analysis result 103, and the artificial intelligence module 112 can be trained accordingly, so that the detailed object type of the corresponding object can be automatically identified in the subsequent analysis.
[0028] Figure 4 FIG. 1 is a schematic diagram of an original image, an analysis result, and a superimposed image according to an embodiment of the present invention. Figure 1 as well as Figure 4 Taking a scene with multiple person images as an example, the original image 101 may be as follows: Figure 4 As shown, the original image 101 may include, for example, multiple character images, some of which are not wearing helmets, and other characters in the multiple character images are wearing helmets. The artificial intelligence module 112 may, for example, perform face recognition on the original image 101 to recognize the faces of the multiple character images, and generate an analysis result 103 having multiple tag information of multiple bounding boxes B1 to B7. In addition, the tag information of the analysis result 103 may also have an object judgment result (not shown) corresponding to each bounding box, such as an object type, an object size, or related object information (such as whether a helmet is worn).
[0029] In this example, the terminal device 130 can select to display only the marking results of the image of the person wearing the helmet according to the user setting or the preset display requirement. Figure 4 The superimposed image 107 shown in the figure may be the result of superimposing the multiple marking information of the bounding boxes B3-B6 on the original image 101. However, in other exemplary embodiments, the terminal device 130 may also directly display the original image 101 or the analysis result 103 according to the user's setting or preset display requirements. In addition, the user may also use the terminal device 130 to annotate the marking information of the bounding boxes B1-B7, such as adding detailed object types such as names of people or names of body parts.
[0030] Figure 5 FIG. 1 is a schematic diagram of the data format of the tag information of an embodiment of the present invention. Figure 1 as well as Figure 5 The marking information described in the embodiments of the present invention may have, for example, Figure 5 The data format of the tag data 500 is shown. The tag data 500 may include a confidence value 501 (i.e., the confidence value generated by the classification model of the artificial intelligence module 112), a font size 502, a tag number 503, a tag type 504 (i.e., the predicted object type result generated by the classification model of the artificial intelligence module 112), and tag location information, wherein the tag location information may include, for example, a bounding box (e.g., Figure 4 1 to 17 ) in the image, but the present invention is not limited thereto. In one embodiment, when the user annotates the marking information 500 of the boundary box through the terminal device 130, the marking number 503 therein can be annotated, for example, the name of a person or the name of a part of a person corresponding to the marking number 503 of the boundary box can be annotated, and the classification model of the artificial intelligence module 112 can generate the analysis result 103 marked with the above-mentioned name of the person or the name of the part of the person according to the annotation of the marking number 503.
[0031] In summary, the image monitoring method, image monitoring system and terminal device of the present invention can enable the terminal device to display the original image, analysis results or superimposed images without delay, so as to provide a good image monitoring effect.
[0032] The above is only a preferred embodiment of the present invention, but it is not intended to limit the scope of the present invention. Any person familiar with this technology can make further improvements and changes on this basis without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined by the claims of this application.
Claims
1. An image monitoring method, characterized in that: include: Obtaining an original image through a camera module, and providing the original image to an artificial intelligence module and a server; Analyzing the original image by the artificial intelligence module to generate an analysis result, and providing the analysis result to the server; Providing the original image and the analysis result to the terminal device through the server; as well as The original image, the analysis result or the superimposed image is displayed by the terminal device, wherein the superimposed image is a result of superimposing a part of the analysis result on the original image.
2. The image monitoring method according to claim 1, characterized in that: The step of providing the original image to the artificial intelligence module and the server also includes: The frames in the original image are numbered by the camera module to synchronously provide the original image and the image numbers corresponding to the frames in the original image to the artificial intelligence module and the server.
3. The image monitoring method according to claim 2, characterized in that: The step of providing the analysis result to the server includes: The artificial intelligence module synchronously provides the analysis result and the image number corresponding to the analysis result to the server.
4. The image monitoring method according to claim 3, characterized in that: The step of providing the original image and the analysis result to the terminal device includes: Synchronizing the original image and the analysis result by the server according to the image number corresponding to the frame in the original image and the image number corresponding to the analysis result; and The server provides the original image, the image number corresponding to the frame in the original image, the analysis result and the image number corresponding to the analysis result to the terminal device.
5. The image monitoring method according to claim 1, characterized in that: The analysis result includes at least one tag information. The at least one mark information includes a mark type, a mark number, a confidence value and a mark position information.
6. The image monitoring method according to claim 1, characterized in that: Also includes: Outputting at least one of the camera module setting parameters and the artificial intelligence module setting parameters to the server through the terminal device; as well as At least one of the camera module setting parameters and the artificial intelligence module setting parameters is outputted to at least one of the camera module and the artificial intelligence module through the server.
7. The image monitoring method according to claim 6, characterized in that: The artificial intelligence module setting parameters include annotation information corresponding to the marked results in the analysis results.
8. The image monitoring method according to claim 1, characterized in that: The camera module and the artificial intelligence module are arranged in an edge device.
9. An image monitoring system, characterized in that: include: Terminal device; A server coupled to the terminal device; an artificial intelligence module, coupled to the server; as well as A camera module is coupled to the artificial intelligence module and the server, The camera module obtains an original image and provides the original image to the artificial intelligence module and the server, and the artificial intelligence module analyzes the original image to generate an analysis result, and provides the analysis result to the server. The server provides the original image and the analysis result to the terminal device, and the terminal device displays the original image, the analysis result or a superimposed image, wherein the superimposed image is a result of superimposing a part of the analysis result on the original image.
10. The video monitoring system according to claim 9, characterized in that: The camera module numbers the frames in the original image to synchronously provide the original image and the image numbers corresponding to the frames in the original image to the artificial intelligence module and the server.
11. The video monitoring system according to claim 10, characterized in that: The artificial intelligence module simultaneously provides the analysis result and the image number corresponding to the analysis result to the server.
12. The video monitoring system according to claim 11, characterized in that: The server synchronizes the original image and the analysis result according to the image number corresponding to the frame in the original image and the image number corresponding to the analysis result, The server provides the original image, the image number corresponding to the frame in the original image, the analysis result and the image number corresponding to the analysis result to the terminal device.
13. The video monitoring system according to claim 9, characterized in that: The analysis result includes at least one tag information. The at least one mark information includes a mark type, a mark number, a confidence value and a mark position information.
14. The video surveillance system according to claim 9, characterized in that: The terminal device outputs at least one of the camera module setting parameters and the artificial intelligence module setting parameters to the server, and the server outputs at least one of the camera module setting parameters and the artificial intelligence module setting parameters to at least one of the camera module and the artificial intelligence module.
15. The video monitoring system according to claim 14, characterized in that: The artificial intelligence module setting parameters include annotation information corresponding to the marked results in the analysis results.
16. The video surveillance system according to claim 9, characterized in that: The camera module and the artificial intelligence module are arranged in an edge device.
17. A terminal device, characterized in that: include: Display unit; A communication interface, coupled to the server, and used to receive the original image and the analysis result; as well as a processing unit coupled to the display unit and the communication interface, and configured to display the original image, the analysis result, or a superimposed image through the display unit, wherein the superimposed image is a result of superimposing a portion of the analysis result on the original image, The original image is provided via a camera module, and the analysis result is provided via an artificial intelligence module based on the original image.
18. The terminal device according to claim 17, characterized in that: The analysis result includes at least one tag information. The at least one mark information includes a mark type, a mark number, a confidence value and a mark position information.
19. The terminal device according to claim 17, characterized in that: The processing unit outputs at least one of the camera module setting parameters and the artificial intelligence module setting parameters to the server through the communication interface, and the server outputs at least one of the camera module setting parameters and the artificial intelligence module setting parameters to at least one of the camera module and the artificial intelligence module.
20. The terminal device according to claim 19, characterized in that Also includes: An input interface coupled to the processing unit, The processing unit obtains the annotation information corresponding to the marked result in the analysis result through the input interface, and the artificial intelligence module sets parameters including the annotation information corresponding to the marked result in the analysis result.
