Image processing method and related device

Abstract

Embodiments of the present application provide an image processing method and related equipment for real-time display of a current picture and corresponding algorithm results. Embodiments of the present application include: obtaining image data to be processed collected by a camera device; performing format conversion on first frame image data and second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and sequentially drawing the first frame display data and the second frame display data on a first frame display layer and a second frame display layer of a target display canvas; when the first frame display data of the first frame display layer has been analyzed based on a picture algorithm and the second frame display data is pushed to a user, algorithm result data corresponding to the first frame display data is drawn on a result display layer, so that the user watches the first frame display data and the algorithm result data on a display screen.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of picture display, and in particular, to an image processing method and related device. BACKGROUND

[0002] The camera equipment in people's work and life has changed from the traditional single intelligent device to the coexistence of multiple device functions. However, the camera picture is usually collected at a frequency of 30 frames per second, that is, about 33.3 milliseconds for collecting a frame. Since the picture algorithm processing component has a high time consumption, greater than 33 milliseconds, if not handled, when the picture rendering result is displayed to the user, the picture algorithm is still processing the frame data, and the user cannot get the algorithm data, which will cause the user to see the frame picture while not being able to see the information corresponding to the frame, such as the number of danger signals, position, high-risk picture warning, etc., seriously affecting the processing accuracy of the alarm system.

[0003] The general processing scheme for this problem is to simultaneously display the current picture and the warning information N seconds ago to the user. This scheme will cause the warning information and the picture to not correspond one by one, and in serious cases, it will affect the misjudgment of the staff and cause dangerous accidents. SUMMARY

[0004] Embodiments of the present application provide an image processing method and related device for real-time display of the current picture and the corresponding algorithm result, so that the algorithm result corresponds to the current picture.

[0005] The first aspect of the embodiments of the present application provides an image processing method applied to an image processing device connected with a camera equipment, and the method comprises:

[0006] obtaining image data to be processed collected by the camera equipment; wherein the image data to be processed at least includes first frame image data and second frame image data, and the second frame image data is the next frame image data of the first frame image data;

[0007] performing format conversion on the first frame image data and the second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and sequentially drawing the first frame display data and the second frame display data on a first frame display layer and a second frame display layer of a target display canvas; wherein the first frame display layer is the previous layer of the second frame display layer;

[0008] when the first frame display data of the first frame display layer has been analyzed based on the picture algorithm, and when the second frame display data is pushed to the user, the first frame display data and the algorithm result data corresponding to the first frame display data are drawn on the first frame display layer and the result display layer respectively, so that the user watches the first frame display data and the algorithm result data on the display screen; wherein the result display layer is located at the top layer of all display layers of the target display canvas, the first frame display layer is the next display layer of the result display layer, and the algorithm result data is obtained based on the analysis of the first frame display data by the picture algorithm.

[0009] Optionally, if the image data to be processed further includes third frame image data, the method further comprises:

[0010] After the first frame display data is drawn on the first frame display layer, the first frame display data is input to the picture algorithm asynchronously, and the first frame display data is analyzed;

[0011] The second frame display data is drawn on the second frame display layer in sequence, and the third frame display data corresponding to the third frame image data is drawn on the third frame display layer; wherein the third frame image data is the next frame image data of the second frame image data;

[0012] If the first frame display data has been analyzed to obtain the algorithm result data, and the step of displaying the first frame display layer and the result display layer on the display screen respectively is performed.

[0013] Optionally, the algorithm result data comprises:

[0014] The first frame display data is input to the picture recognition algorithm for processing to obtain processing result data;

[0015] The processing result data is analyzed based on the alarm processing algorithm to obtain image recognition data corresponding to the first frame display data;

[0016] It is judged whether there is an alarm signal in the image recognition data;

[0017] If the alarm signal exists in the image recognition data, the area corresponding to the alarm signal in the image recognition data is marked to obtain alarm identification data.

[0018] Optionally, the drawing on the first frame display layer and the result display layer respectively comprises:

[0019] drawing the first frame display data on the first frame display layer, drawing the image recognition data on an information display layer, and drawing the alarm identification data on an identification display layer; wherein the identification display layer is located at the uppermost layer of all display layers of the target display canvas, the information display layer is a next display layer of the identification display layer, and the first frame display layer is a next display layer of the information display layer;

[0020] performing picture encoding on the first frame display layer, the information display layer, and the identification display layer, so that the first frame display data, the image recognition data, and the alarm identification data after picture encoding are displayed on the display screen.

[0021] Optionally, the marking of the region corresponding to the alarm signal in the image recognition data to obtain alarm identification data comprises:

[0022] determining an alarm identification position and an alarm identification size of the alarm signal;

[0023] extracting the image recognition data corresponding to the alarm identification position and the alarm identification size to obtain to-be-identified alarm data;

[0024] determining an alarm picture corresponding to the to-be-identified alarm data in the first frame display data, and performing drawing on the edge of the alarm picture to obtain the alarm identification data.

[0025] Optionally, the image processing device is connected with an audio device, and the method further comprises:

[0026] acquiring sound data collected by the audio device;

[0027] determining a timestamp corresponding to the to-be-processed image data, and encoding the sound data and all display data according to the timestamp to obtain audio-video data; wherein the timestamp is used to maintain audio-visual synchronization of the sound data and the all display data, and the all display data at least includes the first frame display data and the second frame display data;

[0028] pushing the audio-video data to a client end, so that the user watches the audio-video data on the client end.

[0029] Optionally, after the audio-video data is obtained, the method further comprises:

[0030] converting the audio-video data into an audio-video file in a target format, and segmenting the audio-video file according to a date-time condition to obtain audio-video files corresponding to different date-times; wherein all audio-video data are stored on a server end;

[0031] receiving the audio and video playback request initiated by the client; wherein the audio and video playback request comprises a target specified date;

[0032] searching according to the target specified date on the server to determine playback audio and video data corresponding to the target specified date;

[0033] real-time streaming the playback audio and video data to enable the user to watch the playback audio and video data on the client.

[0034] The second aspect of the embodiment of the application provides an image processing system, comprising:

[0035] an acquisition unit configured to acquire image data to be processed collected by the camera device; wherein the image data to be processed comprises at least first frame image data and second frame image data, and the second frame image data is the next frame image data of the first frame image data;

[0036] The acquisition unit is further configured to perform format conversion on the first frame image data and the second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and draw the first frame display data and the second frame display data on a first frame display layer and a second frame display layer of a target display canvas in sequence; wherein the first frame display layer is the previous layer of the second frame display layer.

[0037] a drawing unit configured to draw the first frame display data and algorithm result data corresponding to the first frame display data on the first frame display layer and a result display layer respectively when the first frame display data of the first frame display layer has been analyzed based on a picture algorithm and the second frame display data is pushed to a user, to enable the user to watch the first frame display data and the algorithm result data on a display screen; wherein the result display layer is the uppermost layer of all display layers of the target display canvas, the first frame display layer is the next display layer of the result display layer, and the algorithm result data is obtained based on analysis of the first frame display data by the picture algorithm.

[0038] The image processing system provided by the second aspect of the embodiment of the application is used to execute the image processing method of the first aspect.

[0039] The third aspect of the embodiment of the application provides an image processing device, comprising:

[0040] a central processing unit, a memory, an input and output interface, a wired or wireless network interface and a power supply;

[0041] The memory is a temporary storage memory or a persistent storage memory.

[0042] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the image processing method of the first aspect.

[0043] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, the computer readable storage medium includes instructions, when the instructions run on the computer, make the computer execute the image processing method of the first aspect.

[0044] The fifth aspect of the embodiments of the present application provides a computer program product, the computer program product includes instructions, when the instructions run on the computer, make the computer execute the image processing method of the first aspect.

[0045] From the above technical solutions, the embodiments of the present application have the following advantages: by the image processing method disclosed in the embodiments of the present application, first, the image data to be processed collected by the camera equipment is acquired; wherein, the image data to be processed at least includes first frame image data and second frame image data, the second frame image data is the next frame image data of the first frame image data; second, the first frame image data and the second frame image data are format converted to obtain the first frame display data corresponding to the first frame image data and the second frame display data corresponding to the second frame image data, and the first frame display data and the second frame display data are drawn in the first frame display layer and the second frame display layer of the target display canvas in turn; wherein, the first frame display layer is the upper layer of the second frame display layer; finally, when the first frame display data of the first frame display layer has been completed based on the picture algorithm analysis, and when the second frame display data is pushed to the user, the first frame display data and the algorithm result data corresponding to the first frame display data are drawn in the first frame display layer and the result display layer respectively, so that the user can watch the first frame display data and the algorithm result data on the display screen; wherein, the result display layer is located at the uppermost layer of all display layers of the target display canvas, the first frame display layer is the next display layer of the result display layer, and the algorithm result data is obtained based on the picture algorithm analysis of the first frame display data. Thus, by realizing one-to-one correspondence between the picture display data and the result of picture algorithm analysis, the accuracy of image processing is realized. And the user's misjudgment is reduced as much as possible to reduce the probability of causing dangerous accidents. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0047] Figure 1 A software framework schematic diagram of image processing disclosed by the embodiment of the present application;

[0048] Figure 2 A schematic diagram of the architecture flow of an image processing system disclosed by the embodiment of the present application;

[0049] Figure 3 A schematic diagram of the flow of an image processing method disclosed by the embodiment of the present application;

[0050] Figure 4 A schematic diagram of the flow of another image processing method disclosed by the embodiment of the present application;

[0051] Figure 5 A schematic diagram of the flow of another image processing method disclosed by the embodiment of the present application;

[0052] Figure 6 A timing diagram of a picture and an algorithm disclosed by the embodiment of the present application;

[0053] Figure 7 An element structure diagram of a picture layer disclosed by the embodiment of the present application;

[0054] Figure 8 A picture schematic diagram of a danger signal disclosed by the embodiment of the present application;

[0055] Figure 9 A structural schematic diagram of an image processing system disclosed by the embodiment of the present application;

[0056] Figure 10 A structural schematic diagram of an image processing device disclosed by the embodiment of the present application. DETAILED DESCRIPTION

[0057] The terms "first", "second", "third", "fourth" etc. (if any) in the description and claims of the present application and in the above drawings are used for distinguishing between similar objects, not necessarily for describing a specific sequential or chronological order. It is to be understood that the data so distinguished can be interchanged, under appropriate circumstances, such that the embodiments described herein can operate in other sequences than those depicted or claimed. Furthermore, the terms "comprise", "comprising", "include", "including", and "has", "having" and variants thereof are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises, includes, or has a list of steps or units can not necessarily disclose all of those steps or units, but can include additional steps or units not expressly listed or inherent to such process, method, product, or apparatus.

[0058] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] For easier understanding, please refer to Figure 1 , Figure 1 This is a schematic diagram of a software framework for image processing disclosed in an embodiment of this application.

[0061] Depend on Figure 1 As can be seen, the software framework in this application is divided into four parts: 1. Hardware acquisition layer; 2. Algorithm layer; 3. ECS engine layer; 4. Business interface layer. The ECS engine layer includes entities, components, and systems. Furthermore, the software framework in this application adopts a mature ECS engine architecture, ensuring excellent maintainability and scalability. Because ECS has good compatibility, different hardware, algorithms, and third-party libraries can be dynamically integrated into the software system as components, ensuring compatibility while reducing upgrade and iteration costs.

[0062] Furthermore, by Figure 1In the overall framework, the user layer includes a camera interface, an algorithm interface, a live stream interface, a file interface, etc. The ECS includes various components, such as a camera component, a microphone component, an algorithm component, an encoder component, an image layer processing component, a warning component, a live stream component, etc. The hardware includes a camera hardware (USB camera), a microphone, a graphics processing unit (GPU), etc. The algorithm layer includes various algorithms, such as Adas or QR Scan, etc. The three-party dependent library includes a real-time communication technology WebRTC, a barcode processing library Zxing, a media processor, etc. In simple terms, the image processing device described in the embodiments of the present application can be understood as a device containing the algorithm layer and the ECS engine layer. For the convenience of understanding, the details will be described later.

[0063] To solve the above-mentioned technical problems, and to describe the working mode of the software system in detail, Figure 1 , please refer to Figure 3 , Figure 3 The flowchart of the image processing method disclosed in the embodiments of the present application is shown. It includes steps 301-303.

[0064] 301, acquiring image data to be processed collected by a camera device.

[0065] It should be noted that the present embodiment is applied to an image processing device, which can be understood as an integrated device containing an ECS engine layer or an algorithm layer as shown in Figure 1 . The image processing device is connected to the camera device. It is not difficult to understand that the camera device can be a USB camera as described above. The USB camera and the image processing device can be connected through the USB protocol. For the convenience of understanding, the details will not be described hereinafter.

[0066] Specifically, the image processing apparatus can acquire image data to be processed collected by the camera device. It is understood that the image data to be processed at least includes first frame image data and second frame image data, and the second frame image data is the next frame image data of the first frame image data. It should be noted that the first frame and the second frame in the above are not a limitation on the number of frames of image data, and a limitation on the order of frame images. In this embodiment, for the convenience of description, the first frame and the second frame are defined as the frame images of two consecutive frames. Specifically, the first frame image data is the image data collected by the camera device first, and the second frame image data is the image data of the next frame of the first frame image data.

[0067] Further, in this embodiment, there can also be third frame image data or fourth frame image data, etc., which will not be described one by one here. Among them, the third frame image data or the fourth frame image data, etc. are located after the first frame image data and the second frame image data.

[0068] It should be further noted that in this embodiment, the camera device can acquire the camera picture in real time, and cut the camera picture in the form of video frames, so as to obtain the first frame image data and the second frame image data described in the above.

[0069] 302, format conversion is performed on the first frame image data and the second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and the first frame display data and the second frame display data are sequentially drawn on the first frame display layer and the second frame display layer of the target display canvas.

[0070] Therefore, the first frame image data and the second frame image data are converted in file format, so as to obtain the first frame display data corresponding to the first frame image data and the second frame display data corresponding to the second frame image data. Then, the first frame display data and the second frame display data are sequentially drawn on the first frame display layer and the second frame display layer of the target display canvas. It is understood that in the entire display layer, the first frame display layer is above the second frame display layer.

[0071] In one specific embodiment, the camera component at the algorithm layer can convert the first frame image data and the second frame image data into texture data that can be processed by the GPU, i.e. the first frame display data and the second frame display data described above. Then, the first frame display data and the second frame display data are sequentially drawn on the first frame display layer and the second frame display layer.

[0072] It is not difficult to understand that in one of the implementable technical solutions, when the camera picture is collected, the collected image data (which can be understood as first frame image data at this time, which needs to be converted into first frame display data) is first drawn to layer 0 (i.e. first frame display layer). At this time, layer 0 will not be temporarily pushed to the user for viewing. Then, the next frame picture (which can be understood as second frame image data at this time, which needs to be converted into second frame display data) is drawn to layer 1, and it is not displayed to the user, and so on, until layer X (at this time, X can represent the number of layers that the camera component in the image processing device can process, which is not limited here).

[0073] 303、When the first frame display data of the first frame display layer has been completed based on the picture algorithm analysis, and when the second frame display data is pushed to the user, the first frame display data and the algorithm result data corresponding to the first frame display data are drawn in the first frame display layer and the result display layer respectively, so that the user can watch the first frame display data and the algorithm result data on the display screen.

[0074] Therefore, when the first frame display data of the first frame display layer has been completed based on the picture algorithm analysis, and when the second frame display data is pushed to the user, the first frame display data and the algorithm result data corresponding to the first frame display data can be drawn in the first frame display layer and the result display layer respectively, so that the user can watch the first frame display data and the algorithm result data on the display screen. It is not difficult to understand that the result display layer is located at the top of all display layers of the target display canvas, the first frame display layer is the next display layer of the result display layer, and the algorithm result data is obtained after the first frame display data is analyzed based on the picture algorithm.

[0075] In one specific embodiment, when step 302 is performed, i.e. when the first frame display data is drawn to layer 0, the picture input picture algorithm corresponding to layer 0 is also analyzed. Therefore, in the subsequent drawing process, if the first frame display data has been completed based on the picture algorithm analysis, the algorithm processing result after the algorithm analysis, i.e. the algorithm result data and the picture of layer 0 (i.e. the first frame display data), can be drawn in the first frame display layer and the result display layer respectively. It is not difficult to understand that if the picture of layer 0 has been drawn in step 302 at this time, it does not need to be drawn again, which is not described in detail here. Therefore, the algorithm result data and the picture corresponding to the first frame display data of layer 0 can be displayed on the display screen, so that the user can watch on the display screen. At this time, the picture and the algorithm result correspond accurately.

[0076] According to the above description, in another implementable technical solution, when the first frame display data has been drawn on the first frame display layer, the algorithm result data corresponding to the first frame display data is drawn on the result display layer, so that the user can view the first frame display data and the algorithm result data on the display screen. Details are not described here.

[0077] Further, for the second frame display data, the picture of the second frame display data can be displayed after the display of the first frame display data is completed, and so on.

[0078] According to the image processing method disclosed in the embodiment, the image data to be processed collected by the camera device is acquired first, then the first frame image data and the second frame image data are format-converted to obtain the first frame display data corresponding to the first frame image data and the second frame display data corresponding to the second frame image data, and the first frame display data and the second frame display data are drawn on the first frame display layer and the second frame display layer of the target display canvas in sequence. Finally, when the first frame display data of the first frame display layer has been analyzed based on the picture algorithm and the second frame display data is pushed to the user, the first frame display data and the algorithm result data corresponding to the first frame display data are drawn on the first frame display layer and the result display layer respectively, so that the user can view the first frame display data and the algorithm result data on the display screen. Thus, the data of the picture display and the result of the picture algorithm analysis are one-to-one corresponding, the accuracy of image processing is realized, and the misjudgment of the user is reduced as much as possible to reduce the probability of causing dangerous accidents.

[0079] When the alarm processing component identifies various dangerous signals, the traditional alarm system frames the picture area with lines to display to the user. Since this type of system is usually deployed far away from the detection target in order to cover a larger area, the detected dangerous area is relatively small, so the dangerous signal display method of this solution is not easy to attract the attention of the user, and the signal is easy to be ignored, thereby causing a safety accident. To facilitate the detailed description of the image processing method described above, the technical problems described above are solved, please refer to Figure 3 The image processing method is described in detail, and the technical problems described above are solved. Please refer to Figure 4 , Figure 4 The flowchart of another image processing method disclosed in the embodiment is shown. It includes steps 401-410.

[0080] 401, acquiring image data to be processed collected by a camera device.

[0081] 402, format-converting the first frame image data and the second frame image data to obtain the first frame display data corresponding to the first frame image data and the second frame display data corresponding to the second frame image data.

[0082] Steps 401-402 in the embodiment are similar to steps 301-302 in the foregoing Figure 3 embodiment, the image capture device will capture the image data to be processed according to the set capture mode. For example, the frequency can be 30 frames per second, or 60 frames per second, or 120 frames per second, which is not limited here. It should be noted that while the image data is being captured, the sound data can also be captured by the Mic device, which can be referred to in the embodiment shown in Figure 5 , which will not be described here. The embodiment will be illustrated with a capture frequency of 30 frames per second.

[0083] Further, in the embodiment, the image processing device is provided with multiple layers, such as a frame layer for processing the picture, an information layer for processing alarm information, etc. For the convenience of understanding, this will be described in detail later.

[0084] For the convenience of understanding and description, please refer to Figure 2 , that is, the display data of different frames is converted into texture data that can be processed by the GPU, that is, the first frame display data or the second frame display data described in the foregoing. If there is third frame image data or fourth frame image data, or even other frame image data, it can also be converted into third frame display data or fourth frame display data, or even other frame display data, which is not limited here.

[0085] 403、After the first frame display data is drawn on the first frame display layer, the first frame display data is asynchronously input to the picture algorithm for analysis.

[0086] Therefore, corresponding to step 402, if there is third frame image data (which can be simply understood as other frame image data except the first frame image data or the second frame image data, that is, other frame image data in the image data to be processed), after the first frame display data is obtained, the first frame display data can be drawn on the first frame display layer, and then the first frame display data is asynchronously input to the picture algorithm for analysis.

[0087] In one specific embodiment, when the picture captured or recorded by the image capture device is captured, the picture is first drawn on the layer 0 (Frame 0 Layer), which will not be temporarily pushed to the user for viewing, and then step 404 is executed. At the same time, the picture corresponding to the layer 0 is sent to the algorithm for asynchronous processing. That is, when the picture corresponding to the layer 0, that is, the first frame display data, is drawn on the layer 0, the picture algorithm can also analyze the first frame display data synchronously, so as to execute step 405 (analysis step).

[0088] 404. Draw the second frame display data onto the second frame display layer in sequence, and draw the third frame display data corresponding to the third frame image data onto the third frame display layer.

[0089] After the first frame of display data is drawn on the first frame display layer, the second frame of display data can be drawn on the second frame display layer, and the third frame of display data, corresponding to the third frame of image data, can be drawn on the third frame display layer. It's easy to understand that the third frame of image data is the image data following the second frame of image data.

[0090] In one specific embodiment, the display data for the second frame is first drawn on the second frame display layer, and then the display data for the third frame is drawn on the third frame display layer. That is, the display data for the current frame is drawn first, and then the display data for the next frame is drawn.

[0091] Furthermore, in this embodiment, the frame layer is set to have layers 0 to 14. That is, after drawing layer 0, the next frame is drawn to layer 1, but not shown to the user, and so on until layer 14. It is easy to understand that layers 0 to 14 are used to cache the most recent 15 frames from the camera, forming a circular queue.

[0092] To better understand the drawing process of displaying data in the first frame, second frame, third frame, or other frames, please refer to [link / reference]. Figure 6 , Figure 6 This is a timing diagram of a screen and algorithm disclosed in an embodiment of this application. Figure 6 It can be seen that the camera device captures images at a frequency of 30 frames per second. Therefore, the time interval between two adjacent layers in layers 0, 1, and up to 5 is 33.3ms.

[0093] 405. Input the first frame display data into the screen recognition algorithm for processing to obtain the processing result data, and analyze the processing result data based on the alarm processing algorithm to obtain the image recognition data corresponding to the first frame display data.

[0094] Therefore, after step 402, the first frame display data is input into the screen recognition algorithm for processing to obtain the processing result data. Based on the alarm processing algorithm, the processing result data is analyzed to obtain the image recognition data corresponding to the first frame display data.

[0095] In one embodiment, the first frame display data collected by the camera device is input into a picture recognition algorithm component for processing, and processing result data is obtained. Then, a warning component analyzes the algorithm result in real time, and image recognition data corresponding to the first frame display data is obtained.

[0096] Further, after the image data is obtained, step 409 is performed, in which the image recognition data is drawn on an information display layer. That is, the recognition information corresponding to the first frame display data is drawn on the information layer. This will be described in detail later.

[0097] 406. Determine whether there is a warning signal in the image recognition data.

[0098] Based on step 405, it is necessary to determine whether there is a warning signal in the image recognition data.

[0099] In one embodiment, the warning component can determine whether there is a drowning or other warning signal or danger signal in the image recognition data. Specifically, whether a part of the identification in the image recognition data triggers a set recognition identification of the warning signal, such as the height of a person entering the water, or the distance between a sharp object and a person, etc. This is not limited here.

[0100] 407. When there is a warning signal in the image recognition data, determine the warning identification position and the warning identification size of the warning signal.

[0101] Thus, when there is a warning signal in the image recognition data, the area corresponding to the warning signal in the image recognition data needs to be marked to obtain warning identification data. Specifically, the warning identification position and the warning identification size in the warning signal can be determined.

[0102] In one embodiment, when the warning component identifies the warning signal, the corresponding part of the picture information is extracted according to the signal position and size, thereby determining the warning identification position and the warning identification size.

[0103] 408. Extract the image recognition data corresponding to the warning identification position and the warning identification size to obtain the to-be-identified warning data, and determine the warning picture corresponding to the to-be-identified warning data in the first frame display data, and draw on the edge of the warning picture to obtain the warning identification data.

[0104] Therefore, based on step 407, image recognition data corresponding to the alarm sign position and alarm sign size are extracted to obtain alarm data to be identified, and the alarm screen corresponding to the alarm data to be identified in the first frame of display data is determined and drawn at the edge of the alarm screen to obtain alarm sign data.

[0105] In one specific embodiment, image recognition data of the location and size of the alarm sign are extracted, and based on the sign location and size, the alarm screen corresponding to the sign location and size in the first frame of display data is determined. This screen is then enlarged to cover the top layer of the screen, and a striking color is drawn at the edge of the dangerous screen to obtain the alarm sign data.

[0106] For easier understanding and description, please refer to Figure 8 , Figure 8 This is a visual illustration of a danger signal disclosed in an embodiment of this application. Figure 8 As shown in this embodiment, after identifying a drowning-related alarm signal, the location and size of the drowning-related marker can be determined, and the corresponding alarm image in the first frame of display data can be found. This alarm image is then enlarged and highlighted with a prominent color. This process yields the relevant alarm marker data.

[0107] 409. Draw the first frame display data on the first frame display layer, draw the image recognition data on the information display layer, and draw the alarm label data on the label display layer.

[0108] Then, the first frame display data can be drawn on the first frame display layer, the image recognition data on the information display layer, and the alarm label data on the label display layer. It's easy to understand that the label display layer is at the top of all display layers on the target display canvas, the information display layer is the next layer below the label display layer, and the first frame display layer is the next layer below the information display layer.

[0109] In one specific embodiment, the alarm processing component draws the identification information (image recognition data) onto the information display layer (i.e., the information layer Info Layer described above), and simultaneously enlarges and highlights the aforementioned danger area onto the identification display layer (i.e., the picture-in-picture layer PIP Layer described above). Meanwhile, the first frame display data has already been drawn onto the first frame display layer (Frame 0 Layer).

[0110] For a better understanding of the specific distribution of each layer, please refer to [link / reference]. Figure 7 Figure 7 This is an element structure diagram of a screen layer disclosed in an embodiment of this application. Figure 7As can be seen, the topmost layer is the PIP Layer, and successively below are the Info Layer, Frame 0 Layer, Frame 1 Layer,..., Frame 14 Layer. From the above, it can be seen that the PIP Layer is the first layer to be drawn, and the Info Layer is the second layer to be drawn, and so on. Figure 7 In combination with Figure 8 As can be seen, the alarm or danger signal picture can be displayed at any eye-catching position.

[0111] 410. encode the first frame display layer, the information display layer and the identification display layer to make the display screen display the first frame display data, the image recognition data and the alarm identification data after being encoded.

[0112] After the above drawing is completed, the first frame display layer, the information display layer and the identification display layer can be encoded to make the display screen display the first frame display data, the image recognition data and the alarm identification data after being encoded.

[0113] In one specific embodiment, the image layer processing component draws the above three layers to the canvas, and simultaneously notifies the encoder component to encode. Thus, the display screen can display the first frame display data, the image recognition data and the alarm identification data after being encoded. For example, for the drowning alarm signal, please refer to Figure 8 . Details are not described here.

[0114] Further, please refer to Figure 6 , assuming that layer 0 has been processed by the algorithm, the algorithm processing result (i.e. the image recognition data and the alarm identification data) and the picture of layer 0 (i.e. the first frame display data) are pushed to the user for viewing, at this time the picture accurately corresponds to the algorithm result. Meanwhile, the pictures of layers 1-14 are successively sent to the algorithm for processing. When layer 1 is pushed to the user for viewing, the algorithm processing result of layer 0 can be simultaneously displayed.

[0115] Further, please refer to Figure 6 , from Figure 6It can be seen that because the picture is rendered 33.3ms per frame, but the algorithm processing is time-consuming, it is assumed that 66.6ms per frame. In this case, layer 0 is first sent to the algorithm for processing, and after waiting for 66.6ms, the algorithm result of layer 0 is obtained together with layer 0 to display to the user, that is, to the display screen. Because the algorithm itself is relatively slow, it can only calculate layer 2 immediately after layer 0 is calculated, and discard the processing of layer 1. There is no corresponding algorithm processing result for layer 1, and only the algorithm result of layer 0 can be displayed to the user together with layer 1 after layer 0 is displayed. Thus, when layer 0 and layer 1 (i.e., first frame display data and second frame display data) are displayed, only the algorithm result corresponding to the first frame display data (algorithm result 0) can be displayed. Similarly, when layer 2 and layer 3 (i.e., third frame display data and fourth frame display data) are displayed, only the algorithm result corresponding to the third frame display data (algorithm result 2) can be displayed. Details of the subsequent steps are not described here. Until the frame display data corresponding to layer 14 is displayed. That is, layer 0 to layer 14 is a circular queue, and the same steps are performed. At the same time, layer 0-layer 14 corresponding frame display data will not be loaded at the same time, but will be loaded in turn, that is, the current layer 0, the next frame is switched to layer 1, and so on. Further, it can also be understood that, for example, after the pictures of layer 0 and layer 1 are displayed, the pictures of layer 0 and layer 1 can be cleared, and then the pictures corresponding to layer 14, i.e., the sixteenth frame display data and the seventeenth frame display data, are loaded, thereby forming a cycle. Details are not described here.

[0116] Through the image processing method disclosed in the embodiment, the alarm information and the picture information are guaranteed to be one-to-one as much as possible, and the picture fluency is not affected. At the same time, when the alarm processing component identifies a dangerous signal, the corresponding part of the picture information is extracted according to the signal position and size, and is enlarged and overlaid on the uppermost layer of the picture, and a prominent color is drawn on the edge of the dangerous picture. The dangerous signal picture can be displayed at any prominent position. Thus, the dangerous area signal focusing processing can be realized, and the dangerous signal processing rate is improved.

[0117] Further, for the requirement of simultaneously pushing the audio and video data to the server and saving to the hard disk, the traditional scheme uses two encoders, one encoder encodes the audio and video data for pushing, and the other encoder encodes the audio and video data for saving to the hard disk. The scheme using two encoders increases resource consumption, causes the device power consumption to rise, and affects the algorithm processing speed. Therefore, the image processing device in the embodiment can also process the sound synchronously. Please refer to Figure 5 , Figure 5 The flowchart of another image processing method disclosed in the embodiment of the present application is shown. It includes steps 501-505.

[0118] 501、acquire sound data collected by the audio device, determine a timestamp corresponding to the image data to be processed, and encode the sound data and all display data according to the timestamp to obtain audio-video data.

[0119] It is not difficult to understand that in the embodiment, the image processing device is also connected with the audio device. The audio device can be understood as the Mic Component described in the foregoing.

[0120] Specifically, the image processing device acquires sound data collected by the audio device, and also needs to determine a timestamp corresponding to the image data to be processed, and encode the sound data and all display data according to the timestamp to obtain audio-video data. It is not difficult to understand that the timestamp is used to ensure that the sound data and all display data are synchronized in sound and picture, and all display data at least includes the first frame of display data and the second frame of display data.

[0121] In one specific embodiment, as described above Figure 4 The encoder encodes the picture, and also receives sound data collected by the Mic Component. Since it is necessary to ensure that the sound data and the display data are in a synchronized state, the timestamps of all sound data and all display data need to be determined. Thus, the encoder encodes the sound data collected by the Mic Component together when encoding the picture, and ensures that the sound and the picture are synchronized through the timestamps, thereby obtaining audio-video data. All display data at least includes the first frame of display data and the second frame of display data. Figure 4 As described above

[0122] 502、push the audio-video data to the client, so that the user watches the audio-video data on the client.

[0123] Then, the audio-video data can be pushed to the client, so that the user watches the audio-video data on the client.

[0124] In one specific embodiment, the encoded audio and video data is first sent to a live streaming component, which pushes the data to a server for distribution to users for viewing. It will be appreciated that the live streaming component is located in the system of the ECS engine layer. Thus, in this embodiment, the encoder can encode the picture data on the frame layer, and then transmit the encoded picture data to the WebRTC (i.e., the live streaming component) or the audio and video multiplexer (Muxer), respectively, to achieve simple picture playback or audio and video playback. It will be appreciated that the picture data transmitted to the WebRTC can be transmitted to the broadcast network for playback. The audio and video data transmitted to the video multiplexer (Muxer) can be transmitted to the storage space (Storage) for storage, so as to be called later. For details, please refer to steps 503-505.

[0125] It should be further noted that in this embodiment, the picture data described above can also be transmitted to the Storage, and the audio and video data can also be transmitted to the Network. Details are not limited here.

[0126] 503. Convert the audio and video data into an audio and video file in a target format, and segment the audio and video file according to a date and time condition to obtain audio and video files corresponding to different dates and times.

[0127] Thus, based on step 501, the audio and video data will also be converted into an audio and video file in a target format, and the audio and video file will be segmented according to a date and time condition to obtain audio and video files corresponding to different dates and times. It will be appreciated that all the audio and video data is stored on the server.

[0128] In one specific embodiment, in the encoder, the audio and video data will be converted into an audio and video file that can be played by standard software or devices, wherein the format of the audio and video file is a target format. For example, it can be in MP4 format or MOV format, etc. Details are not limited here. Then, the audio and video file can be segmented according to the time condition of the audio and video file, to obtain audio and video files with different dates and times.

[0129] Further, the date and time condition can be the year, month, day, and can also be the hour, minute, and second, etc. Details are not limited here. Thus, according to the actual segmentation condition, audio and video files with different dates and times can be obtained. For example, January 1, 2024 or January 2, 2024. It can also be January 1, 12:12 or January 2, 12:13, etc. Details are not limited here.

[0130] 504、receiving the audio-video playback request initiated by the client to search the server according to the target specified date to determine the playback audio-video data corresponding to the target specified date.

[0131] Thus, after the server stores the audio-video files of different dates and times, the server can receive the audio-video playback request initiated by the client in real time, and search the server according to the target specified date to determine the playback audio-video data corresponding to the target specified date. It is not difficult to understand that the audio-video playback request includes the target specified date. In the embodiment, the target specified date is used to indicate the called audio-video file of the date and time.

[0132] In one specific embodiment, when the client receives the audio-video playback request initiated by the user and requests to watch the playback, the software system can find the audio-video file of the specified date in the server and determine that the audio-video file is the playback audio-video data corresponding to the target specified date.

[0133] Further, the client can query the video file of a specific time point through the server, specifically, by clicking a specific time search control on the client or inputting a specific time point or inputting a related file type, etc., to find the video file to be played back, so that the client generates an audio-video playback request according to the search requirement. Thus, the playback audio-video file required by the user can be determined according to the above-mentioned manner.

[0134] 505、pushing the playback audio-video data in real time to enable the user to watch the playback audio-video data on the client.

[0135] Thus, based on step 504, the playback audio-video data can be pushed in real time to enable the user to watch the playback audio-video data on the client.

[0136] In one specific embodiment, after the server confirms the playback audio-video data to be played back, the server can initiate audio-video pushing through a specific component, for example, WebRTC, at this time, the WebRTC can initiate audio-video pushing, and thus, the client can watch the audio-video picture. It is not difficult to understand that the WebRTC can also be understood as the LiveStream Component described above.

[0137] It is not difficult to understand that the playback audio-video data in the embodiment can also be sound data (i.e., a sound file containing only sound) or picture display data (i.e., a picture file containing only picture display without sound). For the convenience of understanding, the embodiment is only exemplified by the audio-video file, but is not a limitation on the requested file or the stored file in the embodiment.

[0138] Through the image processing method disclosed in the embodiment, the audio and video data can be pushed to the server and saved to the hard disk by using an encoder, so that the multiplexing of the encoder is realized, the encoding speed is improved, the resource occupation is reduced, and the realizability of the scheme is improved.

[0139] In combination with the embodiments shown in the foregoing Figure 1 、 Figure 3 、 Figure 4 and Figure 5 , for the sake of briefly describing the architecture flow of the embodiments of the application, please refer to Figure 2 , Figure 2 , which is an architecture flow schematic diagram of an image processing system disclosed in the embodiments of the application.

[0140] As can be seen from Figure 2 , the specific implementation steps of the image processing system in the embodiments of the application are as follows: 1. First, connect to the Camera hardware (USB Camera) through the USB protocol, and acquire the camera picture in real time. 2. The acquired picture data is sent to the picture recognition algorithm component (Algorithm Component) for processing, and the processing result is acquired. 3. The warning processing component (Warning Component) analyzes the algorithm result in real time, draws the recognition information to the information layer (Info Layer), and judges whether a dangerous signal such as drowning occurs, and enlarges and highlights the dangerous area to the picture-in-picture layer (PIP Layer). 4. While steps 3 and 4 are being executed, the picture data also needs to be converted into texture data that can be processed by the GPU, and drawn to the frame layer (Frame Layer). 5. The image layer processing component (Image Layer Component) draws the above three layers to the canvas (Canvas), and notifies the encoding component (Encoder Component) to perform picture encoding. 6. While the encoder encodes the picture, the sound data collected by the microphone component (Mic Component) is also encoded, and the sound is synchronized with the picture through the time stamp. 7. The encoded audio and video data is first sent to the live streaming component (LiveStream Component), pushed to the server and distributed to the user for viewing. At the same time, the data is encapsulated into an MP4 file format and the like according to the requirement, and written into the hard disk for saving, so as to be reviewed subsequently. 8. When the client requests to view the playback, the software system finds the file of the specified date, reads the audio and video data, and pushes the data to the server through the live streaming component (LiveStream Component) again for distribution to the user for viewing.

[0141] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least some of the other steps or steps or stages in other steps.

[0142] Referring to Figure 9 , Figure 9 A structural schematic diagram of an image processing system disclosed in an embodiment of the present application is shown in FIG. 1.

[0143] The acquisition unit 901 is configured to acquire image data to be processed collected by the camera device; wherein the image data to be processed at least includes first frame image data and second frame image data, and the second frame image data is the next frame image data of the first frame image data.

[0144] The acquisition unit 901 is further configured to perform format conversion on the first frame image data and the second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and draw the first frame display data and the second frame display data on a first frame display layer and a second frame display layer of a target display canvas in sequence; wherein the first frame display layer is the previous layer of the second frame display layer.

[0145] The drawing unit 902 is configured to, when the first frame display data of the first frame display layer has been analyzed based on the picture algorithm and the second frame display data is pushed to the user, draw the first frame display data and algorithm result data corresponding to the first frame display data on the first frame display layer and a result display layer, respectively, to enable the user to view the first frame display data and the algorithm result data on the display screen; wherein the result display layer is the uppermost layer of all display layers of the target display canvas, the first frame display layer is the next display layer of the result display layer, and the algorithm result data is obtained after the first frame display data is analyzed based on the picture algorithm.

[0146] Exemplarily, when the image data to be processed further includes third frame image data, the system further includes an input unit 903.

[0147] The input unit 903 is configured to input the first frame display data to the picture algorithm asynchronously after the first frame display data is drawn on the first frame display layer, and analyze the first frame display data;

[0148] The drawing unit 902 is further configured to draw the second frame display data on the second frame display layer and draw third frame display data corresponding to third frame image data on a third frame display layer in sequence, the third frame image data being next frame image data of the second frame image data;

[0149] The obtaining unit 901 is further configured to obtain algorithm result data when the analysis of the first frame display data is completed, and perform the step of displaying the first frame display layer and the result display layer on the display screen respectively.

[0150] The system further includes a judgment unit 904, for example.

[0151] The obtaining unit 901 is specifically configured to input the first frame display data to the picture recognition algorithm for processing, and obtain processing result data.

[0152] The obtaining unit 901 is further configured to analyze the processing result data based on the alarm processing algorithm, and obtain image recognition data corresponding to the first frame display data.

[0153] The judgment unit 904 is configured to judge whether there is an alarm signal in the image recognition data.

[0154] The obtaining unit 901 is further configured to mark a region corresponding to the alarm signal in the image recognition data when there is an alarm signal in the image recognition data, and obtain alarm identification data.

[0155] The system further includes an encoding unit 905, for example.

[0156] The drawing unit 902 is specifically configured to draw the first frame display data on the first frame display layer, draw the image recognition data on an information display layer, and draw the alarm identification data on an identification display layer, the identification display layer being the uppermost layer of all display layers of the target display canvas, the information display layer being the next display layer of the identification display layer, and the first frame display layer being the next display layer of the information display layer.

[0157] The encoding unit 905 is configured to perform picture encoding on the first frame display layer, the information display layer and the identification display layer, so that the first frame display data, the image recognition data and the alarm identification data after picture encoding are displayed on the display screen.

[0158] The system further includes a determination unit 906 and an extraction unit 907, for example.

[0159] The determining unit 906 is configured to determine a warning mark position and a warning mark size of the warning signal;

[0160] The extracting unit 907 is configured to extract image recognition data corresponding to the warning mark position and the warning mark size, to obtain the to-be-marked warning data.

[0161] The determining unit 906 is further configured to determine a warning picture corresponding to the to-be-marked warning data in the first frame display data, and perform drawing on an edge of the warning picture, to obtain the warning mark data.

[0162] The system also includes, for example, a pushing unit 908.

[0163] The acquiring unit 901 is further configured to acquire sound data collected by the audio device.

[0164] The determining unit 906 is further configured to determine a timestamp corresponding to the to-be-processed image data, and encode the sound data and all display data according to the timestamp, to obtain audio-video data; wherein the timestamp is used to maintain audio-visual synchronization of the sound data and the all display data, and the all display data at least includes the first frame display data and the second frame display data.

[0165] The pushing unit 908 is further configured to push the audio-video data to the client, so that the user watches the audio-video data on the client.

[0166] The system also includes, for example, a receiving unit 909.

[0167] The acquiring unit 901 is further configured to convert the audio-video data into an audio-video file in a target format, and segment the audio-video file according to a date-time condition, to obtain audio-video files corresponding to different date-times; wherein all the audio-video data are stored on the server.

[0168] The receiving unit 909 is configured to receive an audio-video playback request initiated by the client; wherein the audio-video playback request includes a target specified date.

[0169] The determining unit 906 is further configured to search the server according to the target specified date, to determine playback audio-video data corresponding to the target specified date.

[0170] The pushing unit 908 is further configured to push the playback audio-video data in real time, so that the user watches the playback audio-video data on the client.

[0171] Please refer to Figure 10 The structure schematic diagram of the image processing device disclosed in the embodiments of the present application includes:

[0172] The central processor 1001, the memory 1005, the input and output interface 1004, the wired or wireless network interface 1003 and the power supply 1002;

[0173] The memory 1005 is a temporary storage memory or a persistent storage memory.

[0174] The central processor 1001 is configured to communicate with the memory 1005, and execute the instruction operation in the memory 1005 to perform the image processing method in any one of the embodiments shown in the foregoing 3 to Figure 5

[0175] The embodiment of the present application further provides a chip system, characterized in that the chip system comprises at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used for running a computer program or instruction to execute the image processing method in any one of the embodiments shown in the foregoing 3 to Figure 5

[0176] The embodiment of the present application further provides a computer readable storage medium, which comprises instructions, when the instructions are run on a computer, the computer executes the image processing method in any one of the embodiments shown in the foregoing 3 to Figure 5

[0177] The embodiment of the present application further provides a computer program product comprising instructions, when the computer program product is run on a computer, the computer executes the image processing method in any one of the embodiments shown in the foregoing 3 to Figure 5

[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0179] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0180] ​​​​The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0181] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0182] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various program code storage media.

Claims

1. An image processing method, characterized in that, Applied to an image processing device connected to a camera device, the method includes: The image data to be processed is acquired by the camera device; wherein the image data to be processed includes at least a first frame image data and a second frame image data, and the second frame image data is the next frame image data after the first frame image data; The first frame image data and the second frame image data are converted into formats to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, respectively. The first frame display data and the second frame display data are then drawn sequentially on the first frame display layer and the second frame display layer of the target display canvas; wherein, the first frame display layer is the layer above the second frame display layer. When the first frame display data of the first frame display layer has been analyzed based on the image algorithm, and the second frame display data is pushed to the user, the algorithm result data corresponding to the first frame display data is drawn on the result display layer. When pushing the second frame display data, the second frame image data is not analyzed based on the image algorithm, so that the user can view the first frame display data and the algorithm result data on the display screen. The result display layer is located at the top of all display layers of the target display canvas, and the first frame display layer is the next display layer after the result display layer. The algorithm result data is obtained after analyzing the first frame display data based on the image algorithm. Similarly, when pushing display data for odd-numbered frames, the image data of the odd-numbered frames is analyzed based on the image algorithm, so that the user can view the display data of the odd-numbered frames and the algorithm result data on the display screen. When pushing display data for even-numbered frames, the image data of the even-numbered frames is not analyzed based on the image algorithm, and the even-numbered frame image data is discarded.

2. The image processing method according to claim 1, characterized in that, If the image data to be processed also includes a third frame of image data, the method further includes: After the first frame display data is drawn on the first frame display layer, the first frame display data is asynchronously input to the screen algorithm to analyze the first frame display data; The second frame display data is sequentially drawn onto the second frame display layer, and the third frame display data corresponding to the third frame image data is drawn onto the third frame display layer; wherein, the third frame image data is the image data following the second frame image data; If the analysis of the first frame display data has been completed, the algorithm result data is obtained, and the steps of displaying the first frame display layer and the result display layer on the display screen are executed respectively.

3. The image processing method according to claim 2, characterized in that, The data obtained from the algorithm result includes: The first frame of displayed data is input into the screen recognition algorithm for processing to obtain the processing result data; Based on the alarm processing algorithm, the processing result data is analyzed to obtain image recognition data corresponding to the first frame display data; Determine whether an alarm signal exists in the image recognition data; If the alarm signal exists in the image recognition data, the region in the image recognition data corresponding to the alarm signal is marked to obtain alarm identification data.

4. The image processing method according to claim 3, characterized in that, The drawing on the first frame display layer and the result display layer respectively includes: The first frame display data is drawn on the first frame display layer, the image recognition data is drawn on the information display layer, and the alarm identification data is drawn on the identification display layer; wherein, the identification display layer is located at the top of all display layers of the target display canvas, the information display layer is the next display layer after the identification display layer, and the first frame display layer is the next display layer after the information display layer; The first frame display layer, the information display layer, and the identification display layer are image encoded so that the image-encoded first frame display data, the image recognition data, and the alarm identification data are displayed on the display screen.

5. The image processing method according to claim 3, characterized in that, The step of marking the region in the image recognition data corresponding to the alarm signal to obtain alarm identification data includes: Determine the location and size of the alarm indicator for the alarm signal; Extract the image recognition data corresponding to the location and size of the alarm identifier to obtain the alarm data to be identified; The alarm screen corresponding to the alarm data to be identified is determined in the first frame of display data, and drawn on the edge of the alarm screen to obtain the alarm identification data.

6. The image processing method according to claim 1, characterized in that, The image processing device is connected to the audio device, and the method further includes: Acquire the sound data collected by the audio device; A timestamp corresponding to the image data to be processed is determined, and the sound data and all display data are encoded according to the timestamp to obtain audio and video data; wherein, the timestamp is used to ensure audio-visual synchronization between the sound data and all display data, and all display data includes at least the first frame display data and the second frame display data; The audio and video data is pushed to the client so that the user can watch the audio and video data on the client.

7. The image processing method according to claim 6, characterized in that, After obtaining the audio and video data, the method further includes: The audio and video data is converted into audio and video files in the target format, and the audio and video files are segmented according to the date and time conditions to obtain audio and video files corresponding to different dates and times; wherein, all audio and video data are stored on the server. Receive an audio / video playback request initiated by the client; wherein the audio / video playback request includes a target specified date; Search the server according to the target specified date to determine the playback audio and video data corresponding to the target specified date; The playback audio and video data is pushed in real time so that the user can watch the playback audio and video data on the client.

8. An image processing system, characterized in that, The system is applied to an image processing device connected to a camera device, and includes: An acquisition unit is used to acquire image data to be processed collected by the camera device; wherein the image data to be processed includes at least a first frame image data and a second frame image data, and the second frame image data is the next frame image data after the first frame image data; The acquisition unit is further configured to convert the format of the first frame image data and the second frame image data to obtain first frame display data corresponding to the first frame image data and second frame display data corresponding to the second frame image data, and to draw the first frame display data and the second frame display data sequentially on the first frame display layer and the second frame display layer of the target display canvas; wherein, the first frame display layer is the layer above the second frame display layer; A drawing unit is configured to draw the first frame display data and the corresponding algorithm result data on the first frame display layer and the result display layer respectively when the first frame display data of the first frame display layer has been analyzed based on the image algorithm and the second frame display data is pushed to the user. When pushing the second frame display data, the second frame image data is not analyzed based on the image algorithm, so that the user can view the first frame display data and the algorithm result data on the display screen. The result display layer is located at the top of all display layers of the target display canvas, and the first frame display layer is the next display layer after the result display layer. The algorithm result data is obtained after analyzing the first frame display data based on the image algorithm. Similarly, when pushing display data for odd-numbered frames, the image data of the odd-numbered frames is analyzed based on the image algorithm, so that the user can view the display data of the odd-numbered frames and the algorithm result data on the display screen. When pushing display data for even-numbered frames, the image data of the even-numbered frames is not analyzed based on the image algorithm, and the image data of the even-numbered frames is discarded.

9. An image processing apparatus, characterized in that, The device includes: Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the image processing method as described in any one of claims 1 to 7.

Images