Method for identifying display of image frames and related products

Abstract

The application provides a method for identifying and displaying image frames and related products thereof, the method comprising: receiving video data sent from an image acquisition device using a video server; using a computing server to perform: obtaining a video stream, and analyzing and processing the video stream according to an image frame identification method to obtain a plurality of pairs of first time stamps and identification information; and using a client device to perform: receiving the first time stamps and the identification information from the computing server, and receiving the video stream from the video server and obtaining a second time stamp of each image frame in the video stream, determining the identification information corresponding to each image frame according to the first time stamp and the second time stamp, marking a target region on the corresponding image frame according to the identification information, and displaying the image frame after the marking. The scheme of the application can display the analysis result of the image frames in the video stream in real time, and can reduce bandwidth consumption.

Description

Technical Field

[0001] This invention generally relates to the field of identification and display. More specifically, this invention relates to methods and systems for identifying and displaying image frames. Background Technology

[0002] Various AI algorithms require real-time display of computation results during the calculation process. For example, facial recognition algorithms need to outline the area where a person's head is located on each frame of a video stream, while simultaneously displaying information such as facial expressions. Currently, existing solutions for displaying computation results fall into two categories: one is to use the algorithm to output labeled images and push them to the front end for direct display; the other is to convert the video stream into images, label the images using the algorithm, and then compress the labeled images back into a video stream before pushing the labeled video stream to the front end for display. However, using the first type of display solution for real-time display of computation results places high demands on the algorithm's computing power. Furthermore, because uncompressed frame-level images are pushed, the network bandwidth consumption can often be hundreds of times that of directly playing the video stream, making it difficult to use in practical client playback scenarios. Using the second type of display solution for real-time display of computation results requires converting the video stream into images and then back into a video stream, which takes a considerable amount of time. Therefore, there is a need in the existing technology for a real-time labeling and display solution that balances network bandwidth consumption requirements with real-time synchronization requirements. Summary of the Invention

[0003] To address at least one or more of the technical problems mentioned in the background section, this invention proposes a method for identifying and displaying image frames, and related products. The solution of this invention can achieve real-time display of the algorithm results for image frames through timestamp comparison. Therefore, this invention provides solutions in the following aspects.

[0004] According to a first aspect of the present invention, a first method for identifying and displaying an image frame is provided, characterized in that it includes: receiving a plurality of pairs of first timestamps and identification information, wherein the identification information in each pair of first timestamps and identification information is used to identify a target region within an image frame corresponding to the first timestamp of the pair of first timestamps and the first timestamp of the identification information; receiving a video stream and obtaining a second timestamp of each image frame in the video stream; comparing the second timestamp with the first timestamp to obtain a comparison result; and performing a corresponding operation based on the comparison result to identify the target region on the corresponding image frame according to the identification information and display the identified image frame.

[0005] According to a first method for identifying and displaying an image frame according to a first aspect of the present invention, a second method for identifying and displaying an image frame according to a first aspect of the present invention is provided, the method further comprising: initiating an identification display request to a server regarding identifying the image frame; and receiving a first timestamp and identification information from the server as a response to the identification display request.

[0006] According to a first or second method for identifying and displaying an image frame according to a first aspect of the present invention, a third method for identifying and displaying an image frame according to a first aspect of the present invention is provided, wherein performing corresponding operations based on a comparison result includes: determining that the second timestamp matches the first timestamp based on a comparison between the second timestamp and the first timestamp, then identifying the target region on the image frame corresponding to the second timestamp according to the identification information, and displaying the identified image frame.

[0007] According to any one of the first to third claims of the first aspect of the present invention, a method for identifying and displaying image frames is provided, and a fourth method for identifying and displaying image frames according to the first aspect of the present invention is provided, wherein performing corresponding operations based on a comparison result includes: determining, based on a comparison of the second timestamp and the first timestamp, that the reception of identification information of the first image frame is earlier than the first image frame, then caching the first timestamp and identification information corresponding to the first image frame until the first image frame is received; and identifying the target region on the first image frame according to the identification information, and displaying the identified image frame.

[0008] According to any one of the first to fourth claims of the first aspect of the present invention, a fifth method for identifying and displaying image frames according to the first aspect of the present invention is provided, wherein performing corresponding operations based on the comparison result includes:

[0009] Based on the comparison between the second timestamp and the first timestamp, if it is determined that the reception of the identification information of the first image frame is earlier than that of the first image frame, then wait to receive the first image frame within a predetermined first time period; and in response to the failure to receive the first image frame within the predetermined time period, stop displaying the identification of the first image frame and perform an identification display operation on the second image frame, wherein the second image frame is the next frame image of the first image frame.

[0010] According to any one of the first to fifth claims of the first aspect of the present invention, a sixth method for identifying and displaying an image frame is provided, wherein performing a corresponding operation based on a comparison result includes: determining, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, pausing the identification display of the first image frame until the identification information corresponding to the first image frame is received; and identifying the target region on the first image frame according to the identification information, and displaying the identified image frame.

[0011] According to any one of the first to sixth claims of the first aspect of the present invention, a seventh method for identifying and displaying an image frame is provided, wherein performing corresponding operations based on a comparison result includes: determining, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, then waiting to receive the identification information of the first image frame within a predetermined second time period; and responding to the failure to receive the identification information of the first image frame within the predetermined second time period, not identifying the first image frame, directly displaying the first image frame or discarding the first image frame.

[0012] According to any one of the first to seventh methods for identifying and displaying image frames according to the first aspect of the present invention, an eighth method for identifying and displaying image frames according to the first aspect of the present invention is provided, wherein the identification information includes at least one identification information obtained by one or more image frame identification methods, wherein the multiple image frame identification methods include multiple image frame identification methods of the same type and / or multiple image frame identification methods of different types.

[0013] According to any one of the first to eighth methods of identifying and displaying an image frame according to the first aspect of the present invention, a ninth method of identifying and displaying an image frame according to the first aspect of the present invention is provided, wherein the at least one identification information includes information related to identifying the face position, face outline, specific object, facial expression and / or human body key points in the image frame.

[0014] According to any one of the first to ninth claims of the first aspect of the present invention, a tenth method for identifying and displaying an image frame is provided, wherein the identification display request includes selection request information regarding the selection of identification information, the selection request information being used to request the server to send part or all of the at least one identification information.

[0015] According to any one of the first to tenth claims of the first aspect of the present invention, a method for identifying and displaying image frames is provided, wherein the at least one identification information further includes a confidence level associated with the image frame identification method.

[0016] According to any one of the first to eleventh claims of the first aspect of the present invention, a method for marking and displaying an image frame is provided, and a twelfth method for marking and displaying an image frame according to the first aspect of the present invention is further provided, the method comprising: selecting first identification information from the at least one identification information according to a preset selection logic, marking the target region on the image frame according to the first identification information, and displaying the marked image frame; or selecting the first identification information from the at least one identification information according to the confidence level, marking the target region on the image frame according to the first identification information, and displaying the marked image frame.

[0017] According to any one of the first to twelfth claims of the first aspect of the present invention, a method for identifying and displaying image frames is provided, wherein receiving a plurality of pairs of first timestamps and identification information includes: establishing a connection with a server based on a websocket so as to continuously receive a plurality of pairs of first timestamps and identification information from the server.

[0018] According to any one of the first to thirteenth claims of the first aspect of the present invention, a fourteenth method for identifying and displaying image frames is provided, wherein receiving a video stream includes: acquiring a real-time transmitted video stream via web instant messaging (web RTC) technology; and decoding the acquired video stream to obtain the image frames in the video stream.

[0019] According to a second aspect of the present invention, a first client device for identifying and displaying image frames is provided, characterized in that it comprises: a processor; and a memory storing program instructions for identifying and displaying image frames, wherein when the program instructions are executed by the processor, the method according to the first aspect is implemented.

[0020] According to a third aspect of the present invention, a first method for identifying and displaying image frames according to the third aspect of the present invention is provided, comprising: obtaining a first timestamp of each image frame in a video stream; performing corresponding image frame identification method analysis on the video stream according to the image frame identification method to obtain respective identification information for one or more image frames in the video stream, wherein the identification information is used to identify target regions within the image frames; and sending the first timestamp and the identification information to the client device in response to an identification display request from a client device for identifying and displaying image frames.

[0021] According to a first method for identifying and displaying image frames according to a third aspect of the present invention, a second method for identifying and displaying image frames according to a third aspect of the present invention is provided, wherein the identification display request includes selection request information indicating the selection of one or more image frame identification methods, and the method further includes: performing one or more image frame identification method analyses on the video stream according to the selected one or more image frame identification methods to obtain corresponding one or more identification information.

[0022] According to a first or second method for identifying and displaying an image frame according to a third aspect of the present invention, a third method for identifying and displaying an image frame according to a third aspect of the present invention is provided, wherein the one or more identification information further includes a confidence level associated with each.

[0023] According to a fourth aspect of the present invention, a first server for identifying and displaying image frames according to the fourth aspect of the present invention is provided, characterized in that it comprises: a processor; and a memory storing program instructions for identifying and displaying image frames, wherein when the program instructions are executed by the processor, the method according to the third aspect is implemented.

[0024] According to a fifth aspect of the present invention, a first system for identifying and displaying image frames according to the fifth aspect of the present invention is provided, characterized in that it comprises: a video server configured to generate a video stream based on video data acquired by an image acquisition device; a computing server configured to: acquire the video stream, and analyze and process the video stream according to an image frame identification method to obtain a plurality of pairs of first timestamps and identification information, and in response to an identification display request from a client device for identifying and displaying image frames, send the first timestamps and identification information to the client device, wherein the identification information in each pair of first timestamps and identification information is used to identify a target region within an image frame corresponding to the first timestamp of the pair of first timestamps and identification information; and a client device configured to: receive the first timestamps and identification information from the computing server; receive the video stream from the video server and acquire a second timestamp of each image frame in the video stream; determine the identification information corresponding to each image frame based on the first timestamps and second timestamps, identify the target region on the corresponding image frame based on the identification information, and display the identified image frame.

[0025] According to a first system for identifying and displaying image frames according to a fifth aspect of the present invention, a second system for identifying and displaying image frames according to a fifth aspect of the present invention is provided, wherein the client device is further configured to initiate an identification display request to a computing server regarding the identification of the image frame; the computing server is further configured to send a first timestamp and identification information to the client device in response to receiving the identification display request.

[0026] According to a first or second system for identifying and displaying image frames according to a fifth aspect of the present invention, a third system for identifying and displaying image frames according to a fifth aspect of the present invention is provided, wherein the client device is further configured to generate the identification display request in response to receiving a real-time playback request sent by a user, and to send the identification display request to a computing server.

[0027] According to any one of the first to third claims of the fifth aspect of the present invention, a system for identifying and displaying image frames is provided, and a fourth system for identifying and displaying image frames according to the fifth aspect of the present invention is provided, wherein the computing server is further configured to perform image frame identification method analysis on the video stream using multiple image frame identification methods to obtain multiple identification information.

[0028] According to any one of the first to fourth claims of the fifth aspect of the present invention, a fifth system for identifying and displaying image frames is provided, wherein the client device is further configured to generate the identification information selection request in response to receiving a real-time playback request sent by a user, and to send the identification information selection request as part of the identification display request or thereof to a computing server; and the computing server is further configured to select, according to the identification information selection request, to send a first timestamp and some or all of the plurality of identification information to the client device.

[0029] According to any one of the first to fifth claims of the fifth aspect of the present invention, a system for marking and displaying image frames is provided, and a sixth system for marking and displaying image frames according to the fifth aspect of the present invention is further configured to: select first identification information from part or all of the received plurality of identification information according to a preset selection logic, mark the target region on the image frame according to the first identification information, and display the marked image frame; or select the first identification information from part or all of the received plurality of identification information according to a confidence level associated with the identification information, mark the target region on the image frame according to the first identification information, and display the marked image frame.

[0030] According to any one of the first to sixth claims of the fifth aspect of the present invention, a system for identifying and displaying image frames is provided, and a seventh system for identifying and displaying image frames according to the fifth aspect of the present invention is provided, wherein the client device is further configured to: determine that the second timestamp matches the first timestamp based on a comparison between the second timestamp and the first timestamp, then identify the target region on the image frame corresponding to the second timestamp according to the identification information, and display the identified image frame.

[0031] According to any one of the first to seventh claims of the fifth aspect of the present invention, a system for identifying and displaying image frames is provided, and an eighth system for identifying and displaying image frames according to the fifth aspect of the present invention is provided, wherein the client device is further configured to: determine, based on a comparison of the second timestamp and the first timestamp, that the reception of the identification information of the first image frame is earlier than that of the image frame, then cache the first timestamp and the identification information corresponding to the first image frame until the first image frame is received; and identify the target region on the first image frame according to the identification information, and display the identified image frame.

[0032] According to any one of the first to eighth claims of the fifth aspect of the present invention, a ninth system for identifying and displaying image frames is provided, wherein the client device is further configured to: determine, based on a comparison of the second timestamp and the first timestamp, that the reception of the identification information of the first image frame is earlier than that of its image frame, then wait to receive the first image frame for a predetermined first time period; and in response to not receiving the first image frame within the predetermined time period, stop identifying and displaying the first image frame and perform an identification and display operation on a second image frame, wherein the second image frame is the next frame of the first image frame.

[0033] According to any one of the first to ninth claims of the fifth aspect of the present invention, a system for identifying and displaying image frames is provided, and a tenth system for identifying and displaying image frames according to the fifth aspect of the present invention is further configured to: determine, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, then suspend the identification display of the first image frame until the identification information corresponding to the first image frame is received; and identify the target region on the first image frame according to the identification information, and display the identified image frame.

[0034] According to any one of the first to tenth claims of the fifth aspect of the present invention, a system for identifying and displaying image frames is provided, wherein the client device is further configured to: determine, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, then wait to receive the identification information of the first image frame within a predetermined second time period; and, in response to not receiving the identification information of the first image frame within the predetermined second time period, either directly display the first image frame without identifying it or discard the first image frame.

[0035] According to a sixth aspect of the present invention, a first method for identifying and displaying image frames according to the sixth aspect of the present invention is provided, characterized in that it includes: receiving video data sent from an image acquisition device using a video server, and generating a video stream based on the video data; using a computing server to perform: acquiring the video stream, and analyzing and processing the video stream according to an image frame identification method to obtain a plurality of pairs of first timestamps and identification information, wherein the identification information in each pair of first timestamps and identification information is used to identify a target region within an image frame corresponding to the first timestamp of the pair of first timestamps and identification information; and using a client device to perform: receiving the first timestamps and identification information from the computing server, and receiving the video stream from the video server and acquiring a second timestamp of each image frame in the video stream, determining the identification information corresponding to each image frame based on the first timestamps and second timestamps, and identifying the target region on the corresponding image frame based on the identification information, and displaying the identified image frame.

[0036] According to a first method for identifying and displaying an image frame according to a sixth aspect of the present invention, a second method for identifying and displaying an image frame according to a sixth aspect of the present invention is provided, the method further comprising: using the client device to initiate an identification display request to the computing server regarding identifying the image frame; and using the computing server to send the first timestamp and identification information to the client device in response to receiving the identification display request.

[0037] According to the first or second method for identifying and displaying image frames according to the sixth aspect of the present invention, a third method for identifying and displaying image frames according to the sixth aspect of the present invention is provided, wherein the client device is used to perform: generating the identification display request in response to receiving a real-time playback request sent by a user, and sending the identification display request to a computing server.

[0038] According to any one of the first to third claims of the sixth aspect of the present invention, a fourth method for identifying and displaying image frames according to the sixth aspect of the present invention is provided, wherein a computing server is used to perform: in response to receiving a video stream sent by an image acquisition device, performing image frame identification method analysis on the video stream using multiple image frame identification methods to obtain multiple identification information.

[0039] According to any one of the first to fourth claims of the sixth aspect of the present invention, a fifth method for identifying and displaying image frames according to the sixth aspect of the present invention is provided, wherein the client device performs: generating the identification information selection request in response to receiving a real-time playback request sent by a user, and sending the identification information selection request as part of the identification display request to a computing server; and the computing server performs: in response to receiving the identification information selection request, selecting to send a first timestamp and some or all of the plurality of identification information to the client device.

[0040] According to any one of the first to fifth claims of the sixth aspect of the present invention, a sixth method for identifying and displaying an image frame is provided, wherein the client device performs the following: receiving part or all of the plurality of identification information; selecting first identification information from the received part or all of the plurality of identification information according to a preset selection logic; identifying the target region on the image frame according to the first identification information; and displaying the identified image frame; or selecting the first identification information from the received part or all of the plurality of identification information according to a confidence level associated with the identification information; identifying the target region on the image frame according to the first identification information; and displaying the identified image frame.

[0041] According to any one of the first to sixth claims of the sixth aspect of the present invention, a method for identifying and displaying an image frame is provided, wherein the client device performs the following: obtaining a second timestamp; determining that the second timestamp matches the first timestamp based on a comparison between the second timestamp and a first timestamp; identifying the target region on the image frame corresponding to the second timestamp according to the identification information; and displaying the identified image frame.

[0042] According to any one of the first to seventh claims of the sixth aspect of the present invention, a method for identifying and displaying image frames is provided, wherein the client device performs the following: in response to obtaining the second timestamp, determining, based on a comparison of the second timestamp and the first timestamp, that the reception of identification information of the first image frame is earlier than that of its image frame, then caching the first timestamp and the identification information corresponding to the first image frame until the first image frame is received; and identifying the target region on the first image frame according to the identification information, and displaying the identified image frame.

[0043] According to any one of the first to eighth claims of the sixth aspect of the present invention, a ninth method for identifying and displaying image frames according to the sixth aspect of the present invention is provided, wherein the client device performs the following: in response to obtaining the second timestamp, determining, based on a comparison of the second timestamp and the first timestamp, that the reception of the identification information of the first image frame is earlier than that of its image frame, waiting to receive the first image frame for a predetermined first time period; and in response to not receiving the first image frame within the predetermined time period, stopping the identification display of the first image frame and performing an identification display operation on a second image frame, wherein the second image frame is the next frame image of the first image frame.

[0044] According to any one of the first to ninth claims of the sixth aspect of the present invention, a tenth method for identifying and displaying an image frame is provided, wherein the client device performs the following actions: in response to obtaining the second timestamp, determining, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, pausing the identification display of the first image frame until the identification information corresponding to the first image frame is received; and identifying the target region on the first image frame according to the identification information, and displaying the identified image frame.

[0045] According to any one of the first to tenth claims of the sixth aspect of the present invention, a method for identifying and displaying image frames is provided, wherein the client device performs the following: in response to obtaining the second timestamp, determining, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, waiting to receive the identification information of the first image frame within a predetermined second time period; and in response to not receiving the identification information of the first image frame within the predetermined second time period, directly displaying the first image frame without identifying it or discarding the first image frame.

[0046] According to any one of the first to eleventh claims of the sixth aspect of the present invention, a method for identifying and displaying image frames is provided, and a twelfth method for identifying and displaying image frames according to the sixth aspect of the present invention is also provided. The present invention discloses a computer program product, characterized in that it includes a computer program for identifying and displaying image frames, which, when executed by a processor, implements the method according to the first aspect, the method according to the third aspect, or the method according to the sixth aspect.

[0047] According to the methods, apparatus, systems, and computer program products provided in the foregoing aspects of the present invention, it is possible to ensure that the second timestamp pushed via the video stream is synchronized with the first timestamp pushed by the algorithm. This synchronization of timestamps allows the identification information associated with the first timestamp to be loaded into the image frame corresponding to the second timestamp, thereby enabling real-time display of the algorithm results for the image frames. Furthermore, by transmitting the identification information and the video stream independently, when identifying the video stream, not only can network bandwidth consumption and computing power at the client device be reduced, but the identification information can also be dynamically and accurately loaded into the corresponding image frames. Attached Figure Description

[0048] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts wherein:

[0049] Figure 1 This is a system block diagram illustrating the identification and display of image frames according to an embodiment of the present invention;

[0050] Figure 2 This is a flowchart illustrating a method for identifying and displaying image frames according to an embodiment of the present invention;

[0051] Figure 3 This is a flowchart illustrating another method for identifying and displaying image frames according to an embodiment of the present invention;

[0052] Figure 4 This is an exemplary block diagram illustrating a system for identifying and displaying image frames according to an embodiment of the present invention;

[0053] Figure 5 This is an exemplary block diagram illustrating a system for identifying and displaying image frames according to another embodiment of the present invention;

[0054] Figure 6 This is an exemplary block diagram illustrating a system for identifying and displaying image frames according to another embodiment of the present invention;

[0055] Figure 7 This is a flowchart illustrating a system for identifying and displaying image frames according to an embodiment of the present invention; and

[0056] Figure 8 This is an interactive diagram illustrating a system for identifying and displaying image frames according to an embodiment of the present invention. Detailed Implementation

[0057] Embodiments will now be described with reference to the accompanying drawings. It should be understood that, for the sake of simplicity and clarity, reference numerals may be repeated in the drawings to indicate corresponding or similar elements where deemed appropriate. Furthermore, numerous specific details are set forth in this invention to provide a thorough understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein can be practiced without these specific details. In other instances, well-known methods, processes, and components have not been described in detail so as not to obscure the embodiments described herein. Moreover, this description should not be construed as limiting the scope of the embodiments described herein.

[0058] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0059] Figure 1 This is a block diagram illustrating a system 100 for identifying and displaying image frames according to an embodiment of the present invention. In one embodiment, the system 100 may include an image acquisition device (e.g., a camera as shown in the figure) 101, a server 102, and a plurality of client devices 103 (e.g., desktop computers, laptops, and mobile digital assistants as shown).

[0060] In operation, video information (e.g., a work area or a partial street view of a community) can first be acquired in real time by an image acquisition device and sent as a video stream to a server. Each image frame in the video stream can be parsed to obtain a unique presentation time stamp (PTS). Next, after receiving the video stream, the server performs necessary parsing and transcoding to obtain the image frames and their corresponding timestamps (including the first and second timestamps in the context of this invention). Furthermore, the image frames can be transmitted to the algorithm parsing module of server 102, where target detection is performed to obtain identification information for target detection, which is associated with the aforementioned first timestamp.

[0061] In one embodiment, a client device (specifically, its user) can send a command request to the server to display an identifier. In response, the server can send the client device identifier information (the identifier information mentioned above) of the target image frame calculated from the current moment. In addition to receiving the identifier information from the server, the client device also needs to receive the video stream from the server or a playback proxy (not shown in the figure) starting from the current moment. For example, the playback proxy can be integrated with the server or set up independently. Thus, the client device can perform an identifier operation on the corresponding image frame based on the identifier information from the server, for example, by comparing a first timestamp and a second timestamp to determine the corresponding image frame. Then, the client device can display the identifier information on the target area (such as the location or region of interest) of the image frame based on the comparison result. For example, the image frame can be displayed on the client device's LCD screen. In one application scenario, the client device can play the video stream by clicking a web link and display the identifier information on the screen through that web link. Specifically, as shown in Figure 104, the identifier information can, for example, identify the face region in the video stream played on the client device's screen.

[0062] As an example, the data interaction process between the server and the image acquisition device is independent of whether the server receives a command request to display an identifier. Regardless of whether such a command request is received, the image acquisition device can send the acquired video stream to the server. Furthermore, after receiving the video stream, the server can also perform necessary parsing and transcoding to obtain the image frames and corresponding timestamps, and use an algorithm parsing module to perform target detection on the image frames to obtain identifier information for target detection. These two operations are also independent of whether the server receives a command request to display an identifier. That is, regardless of whether a command request to display an identifier is received, as long as the image acquisition device sends the video stream to the server, the server will perform the above-mentioned parsing, transcoding, and target detection operations. Further, after obtaining the identifier information from the target detection in the image frames, the server does not directly push it to the client device. Instead, it repeatedly caches and overwrites it in its memory. Only when the server receives a command request to display an identifier from the client does it send the calculated identifier information to the client device from the moment the command request is received.

[0063] Figure 2This is a flowchart 200 illustrating a method for identifying and displaying image frames according to an embodiment of the present invention. It should be understood that this method can be executed by a client device (e.g., a computer or mobile terminal). For example, the client device can initiate the execution of this method when the image acquisition device begins acquiring image frames. Of course, the present invention does not limit the specific execution time; therefore, it can also be initiated by the user at any time after the image acquisition device has acquired image frames for a period of time, or at any time during the image acquisition process.

[0064] like Figure 2 As shown, firstly, in step S202, an operation is performed to receive multiple pairs of first timestamps and identification information, where each identification information is used to identify a target region within the image frame corresponding to its paired first timestamp. For example, the client device can establish a connection with the server based on a WebSocket to continuously receive multiple pairs of first timestamps and identification information from the server. In one application scenario, the identification information can be used to identify information related to face positions, face contours, specific objects, facial expressions, and / or key points of the human body in the image frame. It can be understood that each image frame of the video stream will be parsed to obtain a timestamp, which is used to authenticate the image time of each frame being captured. For example, the timestamp can be the local time of the image acquisition device capturing the image, the number of milliseconds between the current video stream image frame and the start of video stream playback, and this timestamp can serve as a unique identifier for the synchronous push and display of each image frame of the video stream. Furthermore, the identification information can be obtained by, for example, one or more image frame identification methods, where the multiple image frame identification methods include multiple image frame identification methods of the same type and / or multiple image frame identification methods of different types.

[0065] To facilitate understanding, the following is a brief introduction to various image frame identification methods of the same type and various image frame identification methods of different types, presented as examples. For instance, in the solution provided by this invention, the face recognition method includes three types: face recognition method A, face recognition method B, and face recognition method C. Face recognition method A, face recognition method B, and face recognition method C are thus various image frame identification methods of the same type, all of which are face recognition methods. Furthermore, in the solution provided by this invention, the image frame recognition method includes face recognition methods and pose recognition methods, etc. Face recognition methods and pose recognition methods are thus various image frame recognition methods of different types.

[0066] In one embodiment, before receiving multiple pairs of first timestamps and identification information, an identification display request for an identified image frame can be initiated to the server. This identification display request includes selection request information regarding the selection of identification information and is used to request the server to send part or all of at least one of the identification information. In one application scenario, a first identification information can be selected from at least one identification information according to a preset selection logic. The target region is then identified on the image frame based on the first identification information, and the identified image frame is displayed. For example, the preset selection logic can be set according to different needs, such as selecting different identification information based on different time periods; during the day, identification information obtained using a face recognition method is selected, while at night, identification information obtained using a pose recognition method is selected.

[0067] Furthermore, at least one identification information also includes a confidence level associated with the image frame identification method. In one application scenario, a first identification information can be selected from at least one identification information based on the confidence level, and the target region can be identified on the image frame based on the first identification information, and the identified image frame can be displayed.

[0068] As an example, when the client device receives at least one identification information containing identification information obtained from multiple image frame identification methods of the same type, the identification information is selected based on the confidence level associated with each image frame identification method carried by the identification information. For instance, if the client device receives identification information obtained from face recognition algorithm A, face recognition algorithm B, and face recognition algorithm C, where the confidence level corresponding to face recognition algorithm A is 80%, the confidence level corresponding to face recognition algorithm B is 70%, and the confidence level corresponding to face recognition algorithm C is 90%, then the client device can select the identification information based on the confidence level corresponding to different algorithms, such as selecting the identification information corresponding to face recognition algorithm C with the higher confidence level.

[0069] Returning to the flow, after executing step S202 above, the flow proceeds to step S204. In step S204, the video stream is received and the second timestamp of each image frame in the video stream is obtained (the method of receiving the video stream will be discussed later). Figure 5 and Figure 6 (Detailed description follows), wherein a real-time video stream can be acquired using web real-time communication (web RTC) technology, and the acquired video stream is decoded to obtain image frames from the video stream. Next, in step S204, a second timestamp is compared with a first timestamp to obtain a comparison result, and the process proceeds to step S206. In step S206, corresponding operations are performed based on the comparison result to mark the target region on the corresponding image frame according to the identification information, and the marked image frame is displayed.

[0070] In one embodiment, the second timestamp is determined to match the first timestamp based on a comparison between the second timestamp and the first timestamp. For example, if the server receives the identification information of the 4th second image frame, and the playback agent receives the 4th second image frame, then the second timestamp and the first timestamp match. The identification information of the 4th second from the server can be used to mark the image frame from the playback agent, and the marked image frame can be displayed.

[0071] In one embodiment, if it is determined that the reception of the identification information of the first image frame is earlier than the reception of the first image frame based on a comparison of the second timestamp and the first timestamp, then the first timestamp and identification information corresponding to the first image frame are cached until the first image frame is received, wherein the reception of the first image frame is waited for within a predetermined first time period. Further, in response to the reception of the first image frame within the predetermined time period, a target area is marked on the first image frame according to the identification information, and the marked image frame is displayed. In response to the failure to receive the first image frame within the predetermined time period, the display of the identification information of the first image frame is stopped, and the identification display operation is performed on the second image frame, wherein the second image frame is the next frame image of the first image frame. As an example, suppose the identification information of the 7th second image frame is received from the server, and the image frame of the 5th second is received from the playback agent, while the 7th second image frame has not yet been received. Therefore, the reception of the 7th second identification information is earlier than the reception of the 7th second image frame, and the 7th second identification information and timestamp are cached to wait for the arrival of the 7th second image frame. In one scenario, the client device may not receive the 7th second image frame (i.e., a video stream "packet loss" occurs). In this case, the display of the 7th second image frame can be stopped, and the display of the next image frame can begin.

[0072] In one embodiment, if it is determined, based on a comparison of the second timestamp and the first timestamp, that the reception of the first image frame is earlier than the corresponding identification information, then the display of the identification information for the first image frame is paused until the identification information corresponding to the first image frame is received. This can be done within a predetermined second time period. In response to receiving the identification information corresponding to the first image frame within the predetermined second time period, a target area is marked on the first image frame according to the identification information, and the marked image frame is displayed. In response to not receiving the identification information for the first image frame within the predetermined second time period, the first image frame is not marked, and the first image frame is either directly displayed or discarded. For example, suppose the server receives the identification information for the 5th second image frame, and the playback agent receives the 7th second image frame. Since the identification information for the 7th second image frame has not yet been received, the reception of the 7th second image frame is earlier than the reception of the 7th second identification information, thus pausing the display of the 7th second image frame's identification information to wait for its arrival. In one scenario, if the client does not receive the identification information for the 7th second image frame, it will not mark the 7th second image frame and will directly display the unmarked image frame.

[0073] As an example, if the reception of the first image frame is determined to be earlier than the corresponding identification information based on the comparison between the second timestamp and the first timestamp, the first image frame may not be identified and may be displayed directly.

[0074] Figure 3 This is a flowchart 300 illustrating another method for identifying and displaying image frames according to an embodiment of the present invention. It is understood that this method can be executed by a server. Specifically, the server, as the executor of the method's actions, can be combined with... Figure 2 The described client device works in conjunction with the server to achieve real-time display of the identifier at the client device. In one implementation scenario, the server here can be a dedicated computing server for determining the identifier information and the first timestamp.

[0075] As shown in the figure, in step S302, the first timestamp of each image frame in the video stream is obtained. In one embodiment, the video stream can be an RTSP video stream that accepts the Real-Time Stream Protocol (RTSP). By utilizing the RTSP video stream, the image frames of the video can be controlled, thereby supporting applications with high real-time requirements, such as video chat and video surveillance. Next, in step S304, the video stream is analyzed according to the corresponding image frame identification method to obtain the respective identification information for one or more image frames in the video stream, wherein the identification information is used to identify target regions within the image frames. The image frame identification method described in this invention may include face detection using a target detection algorithm (CenterNet), face bounding box image classification using a lightweight deep convolutional neural network algorithm (VGG), and / or human pose recognition algorithm (Mobilenet-Pose) for human keypoint detection.

[0076] After obtaining the aforementioned identification information, the process proceeds to step S306. In step S306, in response to an identification display request from the client device for identifying image frames, a first timestamp and identification information are sent to the client device. In one embodiment, the identification display request may include selection request information indicating the selection of one or more image frame identification methods, and the video stream may be analyzed using one or more image frame identification methods according to the selected methods to obtain one or more identification information, wherein the one or more identification information also includes a confidence level associated with each. In one application scenario, the identification display request may be a "face recognition" request, in which case the computing server responds to the identification display request by sending "face identification information + timestamp" to the client device. In this case, the face identification information may be bounding box position information used to identify the facial contours in the video frame.

[0077] In another application scenario, the identifier display request can be a request to display "face recognition + pose recognition". In response to this request, the computing server sends "face identifier information + pose identifier information + timestamp" to the client device, allowing the client device to obtain identifier information for face recognition and pose recognition. Therefore, this invention supports flexible switching between multiple algorithms. The solution of this invention allows users to select any one or more of N (N is a positive integer greater than 1) algorithms via the client device. The computing server can then calculate the corresponding multiple identifier information based on the selected algorithm and send it to the client device. Based on this, the user on the client device can select appropriate identifier information (e.g., one with good confidence) from multiple identifier information for real-time identifier display of image frames.

[0078] Figure 4 This is an exemplary block diagram illustrating a system 400 for identifying and displaying image frames according to an embodiment of the present invention. As shown in the figure, the system 400 may include a video server 401, a computing server 402, and a client device 403. It is understood that the video server and the computing server are combined here... Figure 1 The server 102 shown is an example of this system. Through the combined use of the aforementioned video server, computing server, and client device, the system of this invention can perform the identification and display operation of image frames. Alternatively, the video server and computing server can be two independent servers, each performing different operations and having different functions. The operation of the aforementioned video server, computing server, and client device will be described in detail below.

[0079] During operation, a video server 401 can be used to receive signals from an image acquisition device (such as...). Figure 1 The video server 401 receives video data from the image acquisition device 101 and generates a video stream based on the video data, for example, forming a video stream conforming to a streaming media transmission format. In one application scenario, the video server 401 can transmit and play the video stream using, for example, web instant messaging (web RTC) technology. Simultaneously, the computing server 402 can interact with the client device 403 via, for example, a WebSocket long connection, to push multiple pairs of first timestamps and identification information to the client device 403 (as described in the context of this document). Furthermore, the computing server 402 can be used to acquire the video stream and analyze it according to an image frame identification method to obtain multiple pairs of first timestamps and identification information. Specifically, the computing server 402 can perform transcoding and parsing on the acquired video stream by executing a transcoding service to obtain each image frame and first timestamp of the video stream. Then, it performs calculation and analysis on the image frames through algorithmic parsing to obtain identification information for identifying target areas. Finally, in response to an identification display request, the web service sends the identification information and first timestamp obtained from the algorithmic parsing to the client device 403. The following section will discuss the operations of transcoding services, algorithm parsing, and web services within the computing server. Figure 7 Provide a detailed description.

[0080] In one embodiment, a client device 403 can receive a first timestamp and identification information from a computing server 402, and receive a video stream from a video server 401 and obtain a second timestamp for each image frame in the video stream. Further, after receiving the first and second timestamps, the client device 403 can determine the identification information corresponding to each image frame based on the first and second timestamps, and mark a target area on the corresponding image frame according to the identification information to display the marked image frame. In one application scenario, the client device 403 can respond to the user's real-time display request through a display interface, and can then use a browser to receive the first timestamp, second timestamp, and identification information to achieve the marked display of image frames. The description of the client device's browser and display page will be provided below in conjunction with... Figure 7 Provide a detailed description.

[0081] It is important to understand that, in the process of implementing the identification and display operation of image frames, the system of the present invention can use client device 403 to initiate an identification and display request for the identified image frames to computing server 402, and computing server 402 responds by receiving the identification and display request, so as to send the first timestamp and identification information to client device 403. Therefore, computing server 402 only sends the identification information + timestamp of each frame in the current real-time video stream to client device 403 after receiving the identification and display request from client device 403. Similarly, the video stream received by client device 403 is also the real-time video stream after client device 403 sends the identification and display request to video server 401; that is, client device 403 displays real-time video data. In one application scenario, when computing server does not receive the identification and display request from client device 403, it only calculates the identification information + timestamp of each image frame in the received video stream and does not send it to the client device. Furthermore, in this case, client device 403 will not receive video stream data.

[0082] Figure 5 This is an exemplary block diagram illustrating a system 500 for identifying and displaying image frames according to another embodiment of the present invention. Figure 5 As shown, the system 500 may include, in addition to the aforementioned video server 401, computing server 402, and client device 403 (i.e., the system of the present invention for identifying and displaying image frames), also an image acquisition device 501. It is understood that the foregoing has already referenced... Figure 4The video server 401, computing server 402, and client device 403 have been described in detail, so the same content will not be repeated below. In one embodiment, the image acquisition device 501 may be, for example, a camera, used to acquire video data so that the video data can be transmitted to the video server 401 and computing server 402 in the form of a video stream. In one application scenario, the client device 403 receives the second timestamp in the video stream through the conversion by the image acquisition device 501 via the video server 401, that is, the client device 403 obtains the video stream through its first dedicated channel. Secondly, the client device 403 receives the first timestamp and identification information of the image frame through the computing server 402, that is, there is a second dedicated channel for obtaining the first timestamp and identification information.

[0083] Figure 6 This is an exemplary block diagram illustrating a system 600 for identifying and displaying image frames according to another embodiment of the present invention. Figure 6 As shown, the system 600 may include not only the aforementioned image acquisition device 501 and client device 403, but may also include a server 601. It is understood that the preceding text has already referenced... Figure 4 and Figure 5 The image acquisition device 501 and the client device 403 have been described in detail, therefore the same content will not be repeated below. In one embodiment, the server 601 can be used to receive a video stream from the image acquisition device 501, and in response to an identification display request, can be used to send the algorithm-processed identification information and its corresponding first timestamp to the client device 403. Simultaneously, the server 601 can also be used to receive a second timestamp of the video stream image frame in response to the identification display request; that is, in this embodiment, the channel used to transmit the identification information, the first timestamp, and the second timestamp is the same channel. It is understood that the server 601 here may have... Figure 1 The server 102 shown has the same or similar architecture and functions. Furthermore, this server 601 can also be considered as... Figure 5 The invention illustrates a combination of a computing server and a video server. Therefore, the invention imposes excessive restrictions on the specific implementation of the server performing algorithm analysis and the server transmitting real-time video streams; those skilled in the art can make any reasonable arrangement based on the teachings of this invention.

[0084] To facilitate understanding of the system described above for identifying and displaying image frames, the following section will combine... Figure 7 Provide a detailed description of the system displayed by this identifier. Figure 7 This is a detailed flowchart 700 illustrating a system for identifying and displaying image frames according to an embodiment of the present invention.

[0085] like Figure 7 As shown, firstly, in step S702, video data is acquired using an image acquisition device, which can be a camera, and the video data is sent to the video server and the computing server respectively via RTSP video stream. Next, in step S704, the video server receives the RTSP video stream from the image acquisition device and plays it on the client device via Web RTC playback. Simultaneously, in step S706, the computing server can receive the RTSP video stream from the image acquisition device through its transcoding service (as described above). Specifically, the transcoding service can include functions such as receiving and parsing the video stream, decoding the video stream, and sending grayscale and chroma (YUV) data. In one embodiment, the transcoding service, through receiving and parsing the video stream, sequentially parses each image frame of the RTSP video stream using the Real-time Streaming Protocol (RTSP), Real-time Transport Protocol (RTP), and Real-time Transport Control Protocol (RTCP), and then analyzes and judges each image frame of the RTSP video stream to distinguish keyframes, non-keyframes, and parameter frames. At the same time, the first timestamp corresponding to each image frame of the RTSP video stream is also parsed out by using the video stream receiving and parsing operations.

[0086] After completing the aforementioned video stream receiving and parsing operations, the transcoding service module decodes the received image frames into a compressed format through the video stream decoding operation, which is then used to send grayscale and chroma data to the process scheduling center. In one application scenario, the process scheduling center can be used to build image queues, and each image frame can be placed into a corresponding image queue center, which can have different topic categories. Furthermore, different topics can be subscribed to according to different business needs, so that image frames can be consumed passively and then pushed to the algorithm enabled by the computing server. By using the above method, unnecessary performance loss caused by actively pulling image frames can be avoided, and the resource overhead on the business side can be controlled and reduced.

[0087] Returning to the process, after executing step S706, the process proceeds to step S708. In step S708, the input image frame is analyzed using an algorithm to obtain identification information for identifying targets in the image frame, and then the process proceeds to step S710. In step S710, a web backend service provides interfaces for querying camera lists and algorithm lists, and can also provide WebSocket service for client device page connections, thereby forwarding identification information and corresponding first timestamps as needed. Next, in step S712, when the browser decodes and plays each image frame via WebRTC, the current second timestamp is synchronized with the first timestamp pushed by the algorithm, and when the first timestamp and the second timestamp are consistent, the identification information is displayed on the browser's page. In one application scenario, users may view real-time calculation results in any network environment, so WebSocket long connections can be used to optimize network transmission. Furthermore, as a bidirectional communication method, WebSocket can ensure that the algorithm's calculation results are pushed to the browser's page in real time.

[0088] Furthermore, after the WebSocket connection is established, the browser can select the list of available cameras and algorithms provided by the backend server through its display page (as described above), and push the cameras and algorithms to be displayed to the Web backend service module. Upon receiving the aforementioned identifier display request, the Web backend service module filters out the different camera and algorithm identifier information transmitted by the algorithm and pushes the identifier information to the browser. In one application scenario, if the time spent on real-time computation is less than the video playback delay (i.e., the browser receives the image frame identifier information earlier than the image frame itself), the browser can store the received identifier information in its own cache unit. In another application scenario, if the algorithm identifier information is later than the video stream playback, the video playback will be delayed for a fixed duration, which can be set to 1 second to meet the timestamp synchronization requirements.

[0089] Finally, in step S714, while playing the WebRTC video stream, the second timestamp of each image frame is obtained, and the second timestamp is compared with the first timestamp of the identification information in the cache. When the first timestamp is less than or equal to the second timestamp, the identification information in the cache is retrieved and displayed on the browser's display page. In one application scenario, the identification information includes the normalized position, size, shape, color, and identification text of the interface drawing block. Therefore, the browser page can use the canvas drawing tool to draw the identification information onto the video, ultimately achieving the display effect.

[0090] To facilitate understanding of the system described above for identifying and displaying image frames. Figure 8 This is an interactive diagram illustrating a system 800 for identifying and displaying image frames according to an embodiment of the present invention.

[0091] As shown in the figure, this system 800 may include a front-end module 801 (i.e., client device), a back-end service module 802 (i.e., computing server and video server), and a camera 803 (i.e., image acquisition device). First, the server is started at step 808. Next, a transcoding proxy is started at step 809, and a startup message is sent to the camera. Then, at step 810, the acquired video data can be converted into a real-time RTSP video stream using the camera and sent. After completing step 810, the process proceeds to step 811. At step 811, the RTSP video stream from the camera is received, and the video stream is analyzed using algorithms to obtain various identification information for identifying targets detected in the video. Meanwhile, in the front-end module, first, at step 804, the front-end startup operation is performed. Then, at step 805, the camera to be played can be selected from the front-end camera selection list, and an algorithm that meets the target detection requirements can also be selected from the front-end algorithm list. Then, at step 806, the interaction between the front-end module and the back-end service module is achieved using a WebSocket long connection. Returning to the backend service module's process, after executing step 811, the process proceeds to step 812. In step 812, in response to the camera selection request and algorithm selection request from the aforementioned frontend module, the backend service module can filter out the identification information that satisfies the algorithm to obtain the identification information calculated in real time by the algorithm, and then perform the identification information push operation. Next, in step 813, a WebSocket long connection is used to push the identification information and its corresponding first timestamp to the frontend module in real time. Thereafter, the identification information and the first timestamp can be cached in the frontend module's storage unit to wait for the frontend module to receive the image frame corresponding to the identification information.

[0092] Further, after the front-end module completes step 805, in response to the front-end module's identifier display request, the process proceeds to step 807. In step 807, the back-end service module acquires the video stream from the camera by playing WebRTC video and transmits the video stream to the front-end module. Therefore, in step 814, the front-end module can receive each image frame of the video stream from the camera and its associated second timestamp, and synchronize the first and second timestamps to load the identifier information cached in the front-end module into each corresponding image frame of the video stream. After completing step 814, the process proceeds to step 815. In step 815, the front-end module performs the identifier display operation. Finally, in step 816, the process execution ends.

[0093] The method for identifying and displaying image frames according to the present invention has been described in detail above with reference to the accompanying drawings. Based on the above description, those skilled in the art will understand that by using a timestamp comparison method, the identification information can be effectively and accurately loaded onto the corresponding image frames in real time, meeting the requirement for real-time display of calculation results. Furthermore, the method of the present invention avoids directly outputting the identified image frames using algorithms, thereby effectively reducing network bandwidth consumption.

[0094] It should be understood that the possible terms "first" or "second," etc., in the claims, specification, and drawings disclosed in this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0095] It should also be understood that the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of the disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0096] While the embodiments of the present invention are described above, these are merely examples for the purpose of facilitating understanding of the invention and are not intended to limit the scope or application scenarios of the invention. Any person skilled in the art can make any modifications and changes in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the patent protection scope of the present invention shall still be determined by the scope defined in the appended claims.

Claims

1. A system for identifying and displaying image frames, characterized in that, include: A video server configured to generate a video stream based on video data acquired by an image acquisition device; The computing server is configured as follows: The video stream is acquired and analyzed according to the image frame identification method to obtain multiple pairs of first timestamps and identification information. In response to the identification display request from the client device to identify and display the image frame, the first timestamps and identification information are sent to the client device. The identification information in each pair of first timestamps and identification information is used to identify the target area in the image frame corresponding to the first timestamp of the pair of first timestamps and identification information. The client device is configured as follows: Receive the first timestamp and identification information from the computing server; Receive the video stream from the video server and obtain the second timestamp of each image frame in the video stream; The identification information corresponding to each image frame is determined based on the first timestamp and the second timestamp, and the target area is marked on the corresponding image frame according to the identification information, and the marked image frame is displayed.

2. The system according to claim 1, characterized in that, The client device is also configured to send an identification display request to the computing server regarding the identification of the image frame; The computing server is also configured to send the first timestamp and identification information to the client device in response to receiving the identification display request.

3. The system according to claim 1 or 2, characterized in that, The client device is further configured to generate the identifier display request in response to receiving a real-time playback request from a user, and to send the identifier display request to the computing server.

4. The system according to any one of claims 1-3, characterized in that, The computing server is also configured to perform image frame identification method analysis on the video stream using multiple image frame identification methods to obtain various identification information.

5. The system according to any one of claims 1-4, characterized in that, The client device is further configured to generate the identification information selection request in response to receiving a real-time playback request from a user, and to send the identification information selection request as part of the identification display request to the computing server; and The computing server is also configured to select, based on the identification information selection request, to send the first timestamp and some or all of the multiple identification information to the client device.

6. The system according to any one of claims 1-5, characterized in that, The client device is also configured to: According to a preset selection logic, a first identification information is selected from some or all of the received multiple identification information. The target region is identified on the image frame according to the first identification information, and the identified image frame is displayed. or The first identification information is selected from some or all of the received multiple identification information based on the confidence level associated with the identification information. The target region is identified on the image frame based on the first identification information, and the identified image frame is displayed.

7. The system according to any one of claims 1-6, characterized in that, The client device is also configured to: If the comparison between the second timestamp and the first timestamp determines that the second timestamp matches the first timestamp, then the target region is marked on the image frame corresponding to the second timestamp according to the identification information, and the marked image frame is displayed.

8. A method for identifying and displaying image frames, characterized in that, include: The video server receives video data sent from the image acquisition device and generates a video stream based on the video data. The following steps are performed using a computing server: acquiring the video stream and analyzing and processing the video stream according to an image frame identification method to obtain multiple pairs of first timestamps and identification information, wherein the identification information in each pair of first timestamps and identification information is used to identify the target region within the image frame corresponding to the first timestamp of the pair of first timestamps and identification information. as well as The following actions are performed using a client device: receiving the first timestamp and identification information from the computing server, receiving the video stream from the video server and obtaining the second timestamp of each image frame in the video stream, determining the identification information corresponding to each image frame based on the first timestamp and the second timestamp, identifying the target region on the corresponding image frame based on the identification information, and displaying the identified image frame.

9. The method according to claim 8, characterized in that, The following is executed using a computing server: In response to receiving a video stream sent by an image acquisition device, the video stream is analyzed using various image frame identification methods to obtain various identification information.

10. The method according to claim 8 or 9, characterized in that, The client device is used to execute: Receive some or all of the multiple identification information, select the first identification information from the received multiple identification information according to a preset selection logic, mark the target area on the image frame according to the first identification information, and display the marked image frame; or The first identification information is selected from some or all of the received multiple identification information based on the confidence level associated with the identification information. The target region is identified on the image frame based on the first identification information, and the identified image frame is displayed.

Images