Method, device, storage medium and electronic device for determining detection frame information

Abstract

The embodiment of the present invention provides a method, device, storage medium and electronic device for determining detection frame information, wherein the method comprises: acquiring a first frame image; analyzing the first frame image to obtain an analysis result, including a target identifier, a first coordinate and a first time of a first detection frame of a target object, wherein the first time refers to the time when the first frame image is acquired; when it is determined that the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, determining a target prediction rate based on the first coordinate, the first time and the second target information, wherein the second target information is obtained by analyzing the second frame image, wherein the second frame image is an image before the first frame image; determining the third detection frame information of the target object in the third frame image based on the target prediction rate, wherein the third frame image is an image after the first frame image. The present invention solves the problem of mismatch between the trajectory frame and the screen display existing in the related art.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a method, device, storage medium and electronic device for determining detection frame information. Background Art

[0002] At present, the back-end storage hard disk recorder devices in the relevant technologies all support intelligent algorithms such as face, perimeter, and structure. However, since the algorithm computing power and CPU performance of the chip are fixed, and general products also squeeze the capabilities of the chip to the extreme, in scenes with relatively high pressure, the tracking lag of the trajectory frame of the object intelligently identified has become a very common problem and a pain point for many devices. For example, Figure 1 It is an intelligent display solution in related technologies. Figure 1 The current device display scheme and the tracking scheme of the intelligent trajectory frame are described in the text. After the video is collected, the data will be sent to two subsequent stages: data stream 1, which directly displays the collected image in real time, and the time deviation between collection and display can be basically ignored; data stream 2, which sends the data to the algorithm for processing. After the algorithm is processed, the coordinates of the detection frame (or trajectory frame) of the object detected in the image will be returned, and the coordinates of the detection frame will be superimposed on the display image. However, since algorithm processing is a very complex process, especially when the device is under great pressure, the algorithm processing will be very time-consuming, which will result in the detection frame returned by the algorithm corresponding to the detection result of the current display image pushed forward a few seconds, and the detection frame will have serious lag, resulting in the problem of mismatch between the trajectory frame and the display image. In the case of lag in the result returned by the algorithm, how to match the detection frame superimposed on the display image with the image currently being displayed becomes an urgent problem to be solved.

[0003] With respect to the problem of mismatch between the detection frame and the screen display existing in the related art, no effective solution has been proposed yet. Summary of the invention

[0004] Embodiments of the present invention provide a method, device, storage medium and electronic device for determining detection frame information, so as to at least solve the problem of mismatch between the detection frame and the screen display existing in the related art.

[0005] According to an embodiment of the present invention, a method for determining detection frame information is provided, comprising: acquiring a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a target video stream is subjected to predetermined processing; analyzing the first frame image to obtain an analysis result, wherein the analysis result includes target information of a target object, wherein the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate the time when the first frame image was acquired; determining a target prediction rate of the target object based on the first coordinate, the first time, and second target information when it is determined that the target identifier exists in a target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, wherein the second target information is used to indicate information including the target object obtained by analyzing a second frame image, wherein the second frame image is an image acquired before the first frame image, and the target prediction rate is used to indicate a predicted moving rate of the target object after the first time; and determining third detection frame information of the target object in a third frame image based on the target prediction rate, wherein the third frame image is an image acquired after the first frame image.

[0006] In an exemplary embodiment, determining a target prediction rate of the target object based on the first coordinates, the first time and second target information includes: determining the target prediction rate based on the first coordinates, the first time and second coordinates and a second time, wherein the second target information includes the second coordinates and the second time of the second detection frame of the target object, and the second time is used to indicate the time for acquiring the second frame image.

[0007] In an exemplary embodiment, the target predicted rate is determined based on the first coordinate, the first time and the second coordinate and the second time, including: determining the target predicted rate based on the first coordinate, the first time, the second coordinate and the second time according to the following formula: Vx = (x2-x1) ÷ (t2-t1), Vy = (y2-y1) ÷ (t2-t1), wherein (x2, y2) represents the first coordinate, (x1, y1) represents the second coordinate, t2 represents the first time, t1 represents the second time, Vx represents the rate corresponding to the x direction included in the target predicted rate, and Vy represents the rate corresponding to the y direction included in the target predicted rate.

[0008] In an exemplary embodiment, determining third detection frame information of the target object in a third frame image based on the target prediction rate includes: determining a fourth coordinate of the third detection frame based on the target prediction rate, the first time, a third coordinate and a third time, wherein the target information includes the third coordinate of at least one point in the first detection frame, the first coordinate is the coordinate of a predetermined point in the first detection frame, the third time is used to indicate the time when the third frame image is acquired, and the third detection frame information includes the fourth coordinate.

[0009] In an exemplary embodiment, the fourth coordinate of the third detection frame is determined based on the target prediction rate, the first time, the third coordinate and the third time, including: determining the fourth coordinate according to the following formula based on the target prediction rate, the first time, the third coordinate and the third time: Px=(t3-t2)*Vx+X, Py=(t3-t2)*Vy+Y, wherein (X, Y) represents the third coordinate, (Px, Py) represents the fourth coordinate, and t3 represents the third time.

[0010] In an exemplary embodiment, the method further includes: when it is determined that the target identifier does not exist in the target storage unit, creating first record information in the target storage unit, wherein the first record information includes the target identifier, the number of records and a target rate parameter, setting the number of records to a predetermined value, and setting the target rate parameter to 0; when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is less than the predetermined threshold, performing an add 1 operation on the number of records included in the first record information.

[0011] In an exemplary embodiment, the method further includes: when it is determined that first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is greater than or equal to the predetermined threshold, resetting the number of records included in the first record information to a predetermined value, and updating the target rate parameter included in the first record information to the target predicted rate.

[0012] In an exemplary embodiment, the method further includes: when it is determined that the target state information of the target object is in a predetermined state, deleting the first record information, wherein the target information includes the target state information of the target object, and the target state information is used to indicate information that the target object appears, disappears, or is updated.

[0013] In an exemplary embodiment, after determining third detection frame information of the target object in the third frame image based on the target prediction rate, the method further includes: sending the third detection frame information to a target display device to instruct the target display device to determine a third detection frame based on the third detection frame information, superimpose the third detection frame on the third frame image, and display the superimposed image.

[0014] According to another embodiment of the present invention, there is also provided a device for determining detection frame information, comprising: an acquisition module, for acquiring a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a target video stream is subjected to predetermined processing; an analysis module, for analyzing the first frame image to obtain an analysis result, wherein the analysis result includes target information of a target object, and the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate a time when the first frame image is acquired; a first determination module, for determining that the target identifier exists in a target storage unit, and the target storage unit stores the target identifier. When the number of records of the target identification in the storage unit is greater than or equal to a predetermined threshold, a target prediction rate of the target object is determined based on the first coordinate, the first time and the second target information, wherein the second target information is used to indicate information including the target object obtained by analyzing the second frame image, the second frame image is an image captured before the first frame image, and the target prediction rate is used to indicate the predicted movement rate of the target object after the first time; a second determination module is used to determine the third detection box information of the target object in the third frame image based on the target prediction rate, wherein the third frame image is an image captured after the first frame image.

[0015] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0016] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0017] According to the present invention, a first frame image is acquired and analyzed to obtain an analysis result, wherein the analysis result includes a target identifier of a first detection frame of a target object, a first coordinate of the first detection frame, and information of a first time. When it is determined that the target storage unit includes a target identifier and the number of records of the target identifier is greater than or equal to a predetermined threshold, a target prediction rate of the target object is determined based on the first coordinate, the first time, and the second target information, wherein the second target information refers to information including the target object obtained by analyzing a second frame image acquired before the first frame image, and then, based on the target prediction rate, third detection frame information of the target object in a third frame image is determined, wherein the third frame image is an image acquired after the first frame image, that is, the target prediction rate of the target object after the first time is determined according to the target information included in the analysis results of the first frame image and the second frame image, and then, based on the target prediction rate, the detection frame information of the target object in the third frame image is determined. The problem that the detection frame information obtained by analyzing the current frame image lags behind the current display screen due to the time-consuming algorithm processing in the related art is avoided, and the problem that the detection frame does not match the screen display in the related art is solved, thereby achieving the effect of improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an intelligent display solution in the related technology;

[0019] Figure 2 is a hardware structure block diagram of a mobile terminal according to a method for determining detection frame information according to an embodiment of the present invention;

[0020] Figure 3 is a flowchart of a method for determining detection frame information according to an embodiment of the present invention;

[0021] Figure 4 is an overall flow chart of the intelligent display method according to a specific embodiment of the present invention;

[0022] Figure 5 is a flowchart of the return result processing according to a specific embodiment of the present invention;

[0023] Figure 6 4 is a structural block diagram of a device for determining detection frame information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0026] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 2 is a block diagram of the hardware structure of a mobile terminal according to a method for determining detection frame information according to an embodiment of the present invention. Figure 2 As shown, the mobile terminal may include one or more ( Figure 2 Only one is shown in the figure) a processor 202 (the processor 202 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 204 for storing data, wherein the mobile terminal may also include a transmission device 206 and an input / output device 208 for communication functions. It can be understood by those skilled in the art that Figure 2 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown.

[0027] The memory 204 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the detection frame information in the embodiment of the present invention. The processor 202 executes various functional applications and data processing by running the computer program stored in the memory 204, that is, to implement the above method. The memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 204 may further include a memory remotely arranged relative to the processor 202, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 206 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 206 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 206 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] In this embodiment, a method for determining detection frame information is provided. Figure 3 is a flowchart of a method for determining detection frame information according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0030] Step S302, obtaining a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a predetermined processing is performed on a target video stream;

[0031] Step S304, analyzing the first frame image to obtain an analysis result, wherein the analysis result includes target information of the target object, and the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate a time when the first frame image is acquired;

[0032] Step S306, when it is determined that the target identifier exists in the target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, determine a target prediction rate of the target object based on the first coordinate, the first time and second target information, wherein the second target information is used to indicate information including the target object obtained by analyzing a second frame image, the second frame image is an image captured before the first frame image, and the target prediction rate is used to indicate a predicted moving rate of the target object after the first time;

[0033] Step S308: determining third detection frame information of the target object in a third frame image based on the target prediction rate, wherein the third frame image is an image captured after the first frame image.

[0034] Through the above steps, the first frame image is acquired and analyzed to obtain the analysis result, wherein the analysis result includes the target identifier of the first detection frame of the target object, the first coordinates of the first detection frame and the information of the first time. When it is determined that the target storage unit includes the target identifier and the number of records of the target identifier is greater than or equal to the predetermined threshold, the target prediction rate of the target object is determined based on the first coordinates, the first time and the second target information, wherein the second target information refers to the information including the target object obtained by analyzing the second frame image collected before the first frame image, and then the third detection frame information of the target object in the third frame image is determined based on the target prediction rate, wherein the third frame image is an image collected after the first frame image, that is, the target prediction rate of the target object after the first time is determined according to the target information included in the analysis results of the first frame image and the second frame image, and then the detection frame information of the target object in the third frame image is determined based on the target prediction rate. The problem that the detection frame information obtained by analyzing the current frame image lags behind the current display screen due to the time-consuming algorithm processing in the related art is avoided, and the problem that the detection frame does not match the screen display in the related art is solved, thereby achieving the effect of improving the user experience.

[0035] The execution subject of the above steps may be a terminal, a device, or an image processing device, for example, a device capable of analyzing and processing images, or a processor with human-computer interaction capability configured on a storage device, or a processing device or processing unit with similar processing capabilities, but not limited thereto. The following takes the image processing device performing the above operations as an example (only an exemplary description, in actual operation, other devices or modules may also be used to perform the above operations) for description.

[0036] In the above embodiment, the image processing device obtains the first frame image, wherein the first frame image is a frame image included in the target image frame group obtained after the target video stream is processed in a predetermined manner. For example, the first frame image is any frame image after the target video stream is processed. The above predetermined processing may include scaling, frame rate control and other processing. For example, because the algorithm recognizes that there is an optimal resolution, before sending the algorithm frame data, we will scale the captured image to a fixed size before sending it to the algorithm. However, due to limitations such as device computing power, frame rate control may be performed before sending the frame data to the algorithm, and some frames may be discarded. Uniformly extract a part of the frames and send them to the algorithm; analyze the first frame image to obtain the analysis result, for example, send the first frame image to the intelligent algorithm for analysis, and of course send other frame images to the intelligent algorithm for analysis, wherein the analysis result includes the target information of the target object, the target information includes the target identifier of the first detection frame of the target object, the first coordinates of the first detection frame and the first time, the first time is used to indicate the time when the first frame image is collected, for example, the first coordinates can be the coordinates of the center point of the first detection frame, can also be the coordinates of the vertex or corner point of the first detection frame, can also be the coordinates of other points included in the first detection frame;When it is determined that there is a target identifier in the target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, the target prediction rate of the target object is determined based on the first coordinate, the first time and the second target information. For example, when the target identifier (such as ID01) of the first detection frame of the target object is obtained by analyzing the first frame image or other frame images, if the identifier (such as ID01) does not exist in the target storage unit, the record information of the identifier is added to the target storage unit. If the identifier exists in the target storage unit, the number of records of the identifier is managed, for example, the number of records is increased by 1. Taking the above-mentioned predetermined threshold equal to 3 (or 4, or other values) as an example, when the number of records of the target identifier (such as the above-mentioned ID01) recorded in the target storage unit is greater than or equal to 3, the target prediction rate of the target object can be determined based on the first coordinate (such as the coordinate of the center point of the first detection frame), the first time (such as the acquisition time of the first frame image) and the second target information. In practical applications, the target number can also be reset to 0 (or 4) after the number of records reaches the predetermined threshold. 1, or other values), that is, the number of recorded occurrences of the target identifier of the target object is re-counted. After the number of recorded occurrences is reset to 0 (or 1, or other values), in the next statistical period, when the number of recorded occurrences reaches the predetermined threshold again, the target prediction rate of the target object is determined again. Through the above steps, when the same target object appears multiple times by analyzing multiple frames of images, the target prediction rate of the target object can be periodically determined, or the target prediction rate of the target object can be periodically updated, so that the detection frame information of the target object after the first time can be predicted, wherein the second target information is used to indicate the information including the target object obtained by analyzing the second frame of image, and the second target information can also include the identification of the detection frame of the target object (for example, the identification of the second detection frame), coordinates (for example, the second coordinates of the second detection frame) and the second time (such as the acquisition time of the second frame of image), the second frame of image is an image acquired before the first frame of image, and the target prediction rate is used to indicate the predicted movement rate of the target object after the first time;Then, the third detection frame information of the target object in the third frame image is determined based on the target prediction rate, wherein the third frame image is an image captured after the first frame image, that is, the detection frame information of the target object in the third frame image is predicted, and the third frame image may be a frame image currently displayed in the current display screen, and the first frame image is an image captured before the third frame image. Because the time-consuming algorithm processing may cause the result returned by the algorithm (equivalent to the above target information, that is, the relevant information of the first detection frame) to lag relative to the currently displayed frame image (such as the third frame image), and the above steps in this embodiment can avoid the above lag problem by determining the target prediction rate and determining the third detection frame information based on the target prediction rate. In actual applications, after generating the detection frame based on the third detection frame information, the third detection frame is superimposed on the currently displayed frame image, so as to achieve the purpose of matching the detection frame with the screen display. The problem of mismatch between the detection frame and the screen display existing in the related art is solved, and the effect of improving the user experience is achieved. ;

[0037] In an optional embodiment, determining the target prediction rate of the target object based on the first coordinate, the first time and the second target information includes: determining the target prediction rate based on the first coordinate, the first time and the second coordinate and the second time, wherein the second target information includes the second coordinate and the second time of the second detection frame of the target object, and the second time is used to indicate the time of acquiring the second frame of image. In this embodiment, the second target information includes the second coordinate and the second time of the second detection frame of the target object. Of course, the second target information may also include the identifier of the second detection frame of the target object. The identifier of the second detection frame is the same as the target identifier of the first detection frame, and both refer to the same target object (such as a vehicle, or a person, or other object). For example, the first coordinate is the coordinate of the center point of the first detection frame (which may be the coordinate of other points included in the first detection frame), and the second coordinate is the coordinate of the center point of the second detection frame (of course, it may also be the coordinate of other points included in the second detection frame). In this way, the target prediction rate can be determined based on the first coordinate, the first time and the second coordinate and the second time. Through this embodiment, the purpose of determining the target prediction rate according to the analysis results of the first frame of image and the second frame of image is achieved.

[0038] In an optional embodiment, the target predicted rate is determined based on the first coordinate, the first time and the second coordinate and the second time, including: determining the target predicted rate according to the following formula based on the first coordinate, the first time, the second coordinate and the second time: Vx = (x2-x1) ÷ (t2-t1), Vy = (y2-y1) ÷ (t2-t1), wherein (x2, y2) represents the first coordinate, (x1, y1) represents the second coordinate, t2 represents the first time, t1 represents the second time, Vx represents the rate corresponding to the x direction included in the target predicted rate, and Vy represents the rate corresponding to the y direction included in the target predicted rate. In this embodiment, the target prediction rate of the target object in the x direction and the y direction can be calculated according to the formula Vx = (x2-x1) ÷ (t2-t1) and Vy = (y2-y1) ÷ (t2-t1), respectively. The above-mentioned first coordinates can be the coordinates of the center point of the first detection frame, or the coordinates of the vertices or corner points of the first detection frame, or the coordinates of other points included in the first detection frame. Similarly, the above-mentioned second coordinates can be the coordinates of the center point of the second detection frame, or the coordinates of the vertices or corner points of the second detection frame, or the coordinates of other points included in the second detection frame. Through this embodiment, the purpose of determining the rates in the x direction and the y direction included in the target prediction rate based on the coordinates of the first detection frame, the acquisition time of the first frame image, the coordinates of the second detection frame, and the acquisition time of the second frame image is achieved.

[0039] In an optional embodiment, determining the third detection frame information of the target object in the third frame image based on the target prediction rate includes: determining the fourth coordinate of the third detection frame based on the target prediction rate, the first time, the third coordinate and the third time, wherein the target information includes the third coordinate of at least one point in the first detection frame, the first coordinate is the coordinate of a predetermined point in the first detection frame, the third time is used to indicate the time of acquiring the third frame image, and the third detection frame information includes the fourth coordinate. In this embodiment, after determining the target prediction rate, the third detection frame information of the target object in the third frame image can be determined based on the target prediction rate, for example, based on the target prediction rate, the first time, the third coordinate and the third time, the fourth coordinate of the third detection frame of the target object in the third frame image is determined. In practical applications, it can be based on the third coordinate of at least one point in the first detection frame, such as the at least one point can include four vertices or corner points of the first detection frame, and / or the center point of the first detection frame, and / or other points included in the first detection frame. In practical applications, the at least one point can also select only two or three of the four vertices of the first detection frame, and the first coordinate is a predetermined point in the first detection frame, which can be one of the at least one point or a point other than the at least one point. Through this embodiment, the purpose of determining the coordinates of the third detection frame of the target object in the third frame image based on the target prediction rate, the first time, the third coordinate and the third time is achieved.

[0040] In an optional embodiment, determining the fourth coordinate of the third detection frame based on the target prediction rate, the first time, the third coordinate and the third time includes: determining the fourth coordinate according to the following formula based on the target prediction rate, the first time, the third coordinate and the third time: Px = (t3-t2) * Vx + X, Py = (t3-t2) * Vy + Y, wherein (X, Y) represents the third coordinate, (Px, Py) represents the fourth coordinate, and t3 represents the third time. In this embodiment, the coordinates in the x direction and y direction included in the fourth coordinate, namely Px and Py, can be determined respectively according to the formulas Px = (t3-t2) * Vx + X, Py = (t3-t2) * Vy + Y. Through this embodiment, the purpose of determining the fourth coordinate of the third detection frame based on the target prediction rate, the acquisition time of the first frame image, the third coordinate of at least one point included in the first detection frame and the acquisition time of the third frame image is achieved.

[0041] In an optional embodiment, the method further includes: when it is determined that the target identifier does not exist in the target storage unit, creating first record information in the target storage unit, wherein the first record information includes the target identifier, the target number of records and a target rate parameter, setting the target number of records to a predetermined value, and setting the target rate parameter to 0; when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the target number of records included in the first record information is less than the predetermined threshold, performing an add 1 operation on the target number of records included in the first record information. In this embodiment, when it is determined that the target identifier of the first detection frame does not exist in the target storage unit, the first record information is newly created in the target storage unit, that is, when the target identifier of the first detection frame of the target object is detected for the first time, the first record information is established in the target storage unit, and the first record information may include the target identifier, the target number of records and the target rate parameter, wherein the target number of records may be set to a predetermined value, such as 0 or 1 or other values, and the target rate parameter may be set to 0. At this time, the target rate parameter may be used as a target prediction rate to determine or predict the third detection frame information; when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the target number of records of the target identifier is less than a predetermined threshold (such as 3, or 4, or other values), at this time, the target number of records included in the first record information is increased by 1. Through this embodiment, the purpose of establishing the first record information in the target storage unit or updating the target number of records included in the first record information according to the analyzed number of occurrences of the target identifier is achieved.

[0042] In an optional embodiment, the method further includes: when it is determined that the target storage unit has established the first record information and the target identifier is included in the first record information, and the number of records included in the first record information is greater than or equal to the predetermined threshold, the number of records included in the first record information is reset to a predetermined value, and the target rate parameter included in the first record information is updated to the target predicted rate. In this embodiment, when the target storage unit includes the first record information and the number of records of the target identifier included in the first record information is greater than or equal to the predetermined threshold (such as 3, or 4, or other values), the number of records included in the first record information is reset to a predetermined value (such as 0 or 1 or other values), and the target rate parameter included in the first record information is also updated to the above-mentioned target predicted rate, that is, as the number of occurrences of the target identifier obtained by analysis increases, the number of records is updated according to a predetermined period, and the target rate parameter is updated periodically. Through this embodiment, the purpose of updating the number of records and the target rate parameter included in the first record information is achieved.

[0043] In an optional embodiment, the method further includes: when it is determined that the target state information of the target object is in a predetermined state, deleting the first record information, wherein the target information includes the target state information of the target object, and the target state information is used to indicate the information that the target object appears, disappears, or is updated. In this embodiment, the target information may also include the target state information of the target object, such as the state information that the target object appears, disappears, or is updated, and when it is determined that the target state information is in a predetermined state, the first record information in the target storage unit is deleted. For example, when it is determined that object A has disappeared according to a certain frame image (for example, object A has appeared in other frame images before the frame image), the first record information corresponding to the object A can be deleted. Through this embodiment, the purpose of timely cleaning up the information in the target storage unit is achieved, and the effect of saving storage resources is achieved.

[0044] In an optional embodiment, after determining the third detection frame information of the target object in the third frame image based on the target prediction rate, the method further includes: sending the third detection frame information to the target display device to instruct the target display device to determine the third detection frame based on the third detection frame information, superimpose the third detection frame on the third frame image, and display the superimposed image. In this embodiment, after determining the third detection frame information, the third detection frame information can be sent to the target display device to instruct the target display device to generate a third detection frame according to the third detection frame information and superimpose the third detection frame with the third frame image to display the superimposed image. It should be noted that the target display device displays in real time the image frame group obtained after the predetermined processing of the aforementioned target video stream. For example, the third frame image mentioned above is the frame image currently being displayed by the display device. After the third detection frame is superimposed on the third frame image, the display screen can simultaneously display the current frame image and the detection frame of the target object included in the current frame image, thereby achieving the purpose of matching the detection frame with the display screen. Of course, the third frame image may also include multiple different objects. Similarly, according to the above steps, the detection frame information of each object included in the multiple different objects is determined and superimposed on the third frame image, so that the detection frames of multiple different objects can be displayed simultaneously in the current display screen. Through this embodiment, the purpose of determining the third detection frame based on the predicted third detection frame information and superimposing the third detection frame on the currently displayed frame image is achieved.

[0045] Obviously, the above described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The present invention will be specifically described below in conjunction with the embodiments.

[0046] The basic idea of ​​this embodiment is to bring the frame timestamp of the current frame sent to the algorithm into the frame through a private pointer before sending it to the intelligent algorithm for analysis. After the algorithm analysis, it returns the trajectory frame and other results, processes the trajectory frame results, and predicts the trajectory frame of the current preview display frame. The core part is the result processing and trajectory frame prediction, which is described in detail below.

[0047] Figure 4 is an overall flow chart of the intelligent display method according to a specific embodiment of the present invention, the process includes:

[0048] Step 1: Acquisition, scaling, and frame rate control

[0049] All of them are currently enabled with intelligent support.

[0050] Collection: refers to the data collected by the front-end camera and input to the back-end device through AD conversion;

[0051] Scaling: Because the algorithm recognition has an optimal resolution, we will scale the captured image to a fixed size before sending the algorithm frame data;

[0052] Frame rate control: Due to limitations in device computing power, frame rate control is performed before sending frame data to the algorithm. Some frames are discarded and some frames are evenly extracted to send to the algorithm.

[0053] It should be noted that after the video is collected, the data will be sent to two subsequent stages, such as Figure 4 In step 1, data stream 1 and data stream 2 are processed, such as the above-mentioned scaling, frame rate control, etc.

[0054] Step 2: Print the current timestamp

[0055] When sending an intelligent algorithm, the frame data sent in can be brought in with the frame information, and the frame information will be brought out when the algorithm result is returned. Here, the timestamp of the current frame acquisition can be added to the frame information when sending the algorithm, and brought out when the algorithm result is returned. The role of the timestamp will be introduced in detail in the following steps.

[0056] Step 3: Algorithm Analysis

[0057] After the collected frame data is sent to the algorithm, the algorithm will perform intelligent analysis on the current frame, identify the objects in the frame, which can be people, cars, animals, etc., and return the object's ID, object status (appearance, disappearance, update), and the coordinate frame where the object is located, that is, the trajectory frame mentioned above.

[0058] Step 4: Result processing

[0059] The following is a detailed description of the algorithm return result processing process, such as Figure 5 As shown, Figure 5 1 is a flowchart of the return result processing according to a specific embodiment of the present invention, and the process includes:

[0060] S502, the algorithm result is returned, and the object management linked list (corresponding to the aforementioned target storage unit) is searched to see whether the current object ID (corresponding to the aforementioned target identifier) ​​exists;

[0061] S504, if it does not exist, then add management of the object ID;

[0062] S506, and save the following information corresponding to the current object id into the linked list for management:

[0063] 1) Calculate and save the center coordinates (x1, y1) of the current object trajectory box;

[0064] 2) Save the pts1 of the current frame (pts is the timestamp, which is the frame information brought in when sending the algorithm for analysis);

[0065] 3) Set the Count of the current id to 0 (Count is used for counting, indicating the number of frames in which the current object appears continuously, and Count is equivalent to the aforementioned target number);

[0066] 4) Set the current horizontal and vertical coordinate movement rate Vx=0, Vy=0;

[0067] S508, using the current track frame, and then entering S518 for display;

[0068] S510, in the above step S502, if the search finds that the current object id already exists, determine whether the Count value is greater than or equal to 3 (the introduction of Count is to update the object movement rate in real time, and the object movement rate is updated once the object appears for 3 consecutive frames).

[0069] The above step S510 may also be configured to determine whether the Count value is greater than or equal to other values, such as 4 or 5, or other values.

[0070] S512, if Count is greater than or equal to 3, recalculate the object's movement speed according to the following formula 5.1:

[0071]

[0072] The relevant parameters or symbols in the above formula are explained as follows:

[0073] ① The coordinates of the center point of the object saved 3 frames ago: (x1, y1) (corresponding to the second coordinate mentioned above);

[0074] ② The coordinates of the center point of the object trajectory frame in the current frame (x2, y2) (corresponding to the first coordinates mentioned above);

[0075] ③ The acquisition frame timestamp corresponding to the result frame returned by the algorithm 3 frames ago: pts1 (corresponding to the aforementioned second time);

[0076] ④ The current algorithm returns the acquisition frame timestamp corresponding to the result frame: pts2 (corresponding to the aforementioned first time);

[0077] S514, after calculating the new rate, the following information needs to be updated and saved in the linked list corresponding to the object ID:

[0078] 1) Save the center coordinates of the trajectory box of the current object as (x1, y1), that is, replace the center coordinates of the trajectory box (x1, y1) corresponding to the object id in the linked list with the center coordinates of the trajectory box of the current object (x2, y2);

[0079] 2) Save the acquisition frame timestamp corresponding to the current frame as pts1, that is, replace the acquisition frame timestamp pts1 corresponding to the object id in the linked list with the acquisition frame timestamp pts2 corresponding to the current frame;

[0080] 3) Save the current horizontal and vertical coordinate movement rates Vx and Vy;

[0081] 4) Change Count to 0 and save.

[0082] It should be noted that, in the above steps S506 and S514, Count may also be set to 1 or other values.

[0083] S516, if Count is not greater than or equal to 3, then Count+1 (or Count++) is added, and the saved object motion rate is used to perform trajectory frame prediction.

[0084] In the above steps S502-S516, the prediction method of the object movement rate is determined according to different situations, so as to predict the trajectory frame for the subsequent steps.

[0085] Step 5: Trajectory box prediction

[0086] (1) When the situation in step S504 in Step 4 occurs: no prediction is performed, and the trajectory box reported by the current algorithm is directly used for display (corresponding to the aforementioned step S508);

[0087] (2) When the situation in step S512 of Step 4 occurs: make a prediction, and use the newly calculated motion rate to predict the trajectory frame according to the following formula 5.2;

[0088] (3) When the situation in step S516 of Step 4 occurs: make a prediction, according to the following formula 5.2, using the saved motion rate to predict the trajectory frame;

[0089]

[0090] The relevant parameters or symbols in the above formula are explained as follows:

[0091] ① The timestamp of the frame currently being displayed on the preview screen is: pts3 (i.e. the current system time);

[0092] ②The timestamp of the acquisition frame corresponding to the result frame returned by the current algorithm: pts2;

[0093] ③ The current horizontal and vertical coordinate movement rates Vx and Vy;

[0094] ④x, y: represents the horizontal and vertical coordinates of the four points of the object returned in the current frame, for example, the horizontal and vertical coordinates of the four corner points (or vertices) of the object's trajectory box (or target detection box), or the coordinates of the center point of the trajectory box (or target detection box);

[0095] Step 6: Overlay display

[0096] Use the predicted coordinates calculated in Step 5 (i.e. the predicted x-coordinate and predicted y-coordinate calculated by the above formula 5.2) to overlay the trajectory frame of the display interface.

[0097] Step 7: Delete the object id

[0098] When the algorithm returns that the state of the current object id is disappeared, delete the current object id from the management list. This step is Step 7. Figure 4 Not shown.

[0099] In the above embodiment, compared with the solutions in the related technology, the advantages of the embodiments of the present application are: no matter what pressure environment the current device is in, whether the computing power and CPU are in excess, the relative movement rate of the object is unchanged, and the use of the unchanged movement rate and acquisition frame timestamp can accurately predict the location of the trajectory box of the object in the current display frame, thereby improving the experience, and the entire judgment will not occupy too much CPU and memory, while solving a major pain point of the product.

[0100] Through the above embodiment, the motion rate is calculated from the pts of the acquisition frame carried by the algorithm frame and the motion distance equivalent to the difference of the acquisition pts, so as to use the motion rate to predict the distance of the object's motion at the time when the algorithm returns the result relative to the acquisition time, and predict the trajectory frame corresponding to the pts of the current display frame, so as to achieve real-time tracking of the trajectory frame; the lag distance of the changing trajectory is predicted with the real and unchanged motion rate of the object. That is, in the case where there is a lag in the result returned by the algorithm, the purpose of matching the trajectory frame superimposed on the display screen with the currently displayed screen is achieved.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0102] In this embodiment, a device for determining detection frame information is also provided. Figure 6 is a structural block diagram of a device for determining detection frame information according to an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0103] An acquisition module 602 is used to acquire a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a predetermined processing is performed on a target video stream;

[0104] An analysis module 604 is used to analyze the first frame image to obtain an analysis result, wherein the analysis result includes target information of the target object, and the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate a time when the first frame image is acquired;

[0105] A first determination module 606 is configured to determine a target prediction rate of the target object based on the first coordinate, the first time, and second target information when it is determined that the target identifier exists in the target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, wherein the second target information is used to indicate information including the target object obtained by analyzing a second frame image, the second frame image is an image captured before the first frame image, and the target prediction rate is used to indicate a predicted moving rate of the target object after the first time;

[0106] The second determination module 608 is used to determine third detection frame information of the target object in a third frame image based on the target prediction rate, wherein the third frame image is an image captured after the first frame image.

[0107] In an optional embodiment, the above-mentioned first determination module 606 includes: a first determination unit, used to determine the target prediction rate based on the first coordinate, the first time and the second coordinate, and the second time, wherein the second target information includes the second coordinate and the second time of the second detection box of the target object, and the second time is used to indicate the time for acquiring the second frame image.

[0108] In an optional embodiment, the above-mentioned first determination unit includes: a first determination subunit, used to determine the target predicted rate based on the first coordinate, the first time, the second coordinate and the second time according to the following formula: Vx = (x2-x1) ÷ (t2-t1), Vy = (y2-y1) ÷ (t2-t1), wherein (x2, y2) represents the first coordinate, (x1, y1) represents the second coordinate, t2 represents the first time, t1 represents the second time, Vx represents the rate corresponding to the x direction included in the target predicted rate, and Vy represents the rate corresponding to the y direction included in the target predicted rate.

[0109] In an optional embodiment, the second determination module 608 includes: a second determination subunit, used to determine the fourth coordinate of the third detection frame based on the target prediction rate, the first time, the third coordinate and the third time, wherein the target information includes the third coordinate of at least one point in the first detection frame, the first coordinate is the coordinate of a predetermined point in the first detection frame, the third time is used to indicate the time for acquiring the third frame image, and the third detection frame information includes the fourth coordinate.

[0110] In an optional embodiment, the above-mentioned second determination subunit includes: a second determination subunit, used to determine the fourth coordinate based on the target prediction rate, the first time, the third coordinate and the third time according to the following formula: Px=(t3-t2)*Vx+X, Py=(t3-t2)*Vy+Y, wherein (X, Y) represents the third coordinate, (Px, Py) represents the fourth coordinate, and t3 represents the third time.

[0111] In an optional embodiment, the above-mentioned device also includes: a new creation module, which is used to create first record information in the target storage unit when it is determined that the target identifier does not exist in the target storage unit, wherein the first record information includes the target identifier, the number of records and the target rate parameter, the number of records is set to a predetermined value, and the target rate parameter is set to 0; an execution module, which is used to perform an add 1 operation on the number of records included in the first record information when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is less than the predetermined threshold.

[0112] In an optional embodiment, the above-mentioned device also includes: a processing module, which is used to reset the number of records included in the first record information to a predetermined value and update the target rate parameter included in the first record information to the target predicted rate when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is greater than or equal to the predetermined threshold.

[0113] In an optional embodiment, the above-mentioned device also includes: a deletion module, which is used to delete the first record information when it is determined that the target state information of the target object is in a predetermined state, wherein the target information includes the target state information of the target object, and the target state information is used to indicate information that the target object appears, disappears or is updated.

[0114] In an optional embodiment, the above-mentioned device also includes: a sending module, which is used to send the third detection frame information to the target display device after determining the third detection frame information of the target object in the third frame image based on the target prediction rate, so as to instruct the target display device to determine the third detection frame based on the third detection frame information, superimpose the third detection frame on the third frame image, and display the superimposed image.

[0115] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0116] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0117] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0118] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0119] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0120] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0121] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining detection frame information, characterized in that: include: Acquire a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a predetermined process is performed on a target video stream; Analyze the first frame of image to obtain an analysis result, wherein the analysis result includes target information of the target object, and the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate a time when the first frame of image is acquired; When it is determined that the target identifier exists in the target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, determine a target prediction rate of the target object based on the first coordinate, the first time and second target information, wherein the second target information is used to indicate information including the target object obtained by analyzing a second frame image, the second frame image is an image captured before the first frame image, and the target prediction rate is used to indicate a predicted moving rate of the target object after the first time; Determining third detection frame information of the target object in a third frame image based on the target prediction rate, wherein the third frame image is an image acquired after the first frame image; The method further includes: in the case where it is determined that the target identifier does not exist in the target storage unit, creating first record information in the target storage unit, wherein the first record information includes the target identifier, the number of records and a target rate parameter, setting the number of records to a predetermined value, setting the target rate parameter to 0, and using the first detection frame to determine the third detection frame information; The method also includes: when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is less than the predetermined threshold, performing an add-1 operation on the number of records included in the first record information, and determining the third detection frame information based on the target rate parameter recorded in the first record information.

2. The method according to claim 1, characterized in that Determining a target predicted rate of the target object based on the first coordinate, the first time, and the second target information includes: The target prediction rate is determined based on the first coordinate, the first time and the second coordinate and the second time, wherein the second target information includes the second coordinate and the second time of the second detection box of the target object, and the second time is used to indicate the time of acquiring the second frame image.

3. The method according to claim 2, characterized in that Determining the target predicted rate based on the first coordinate, the first time and the second coordinate, the second time includes: The target predicted rate is determined based on the first coordinate, the first time, the second coordinate, and the second time according to the following formula: Vx=(x2-x1)÷(t2-t1), Vy=(y2-y1)÷(t2-t1), Among them, (x2, y2) represents the first coordinate, (x1, y1) represents the second coordinate, t2 represents the first time, t1 represents the second time, Vx represents the rate corresponding to the x direction included in the target prediction rate, and Vy represents the rate corresponding to the y direction included in the target prediction rate.

4. The method according to claim 3, characterized in that Determining third detection frame information of the target object in a third frame image based on the target prediction rate includes: Based on the target prediction rate, the first time, the third coordinate and the third time, the fourth coordinate of the third detection frame is determined, wherein the target information includes the third coordinate of at least one point in the first detection frame, the first coordinate is the coordinate of a predetermined point in the first detection frame, the third time is used to indicate the time of acquiring the third frame image, and the third detection frame information includes the fourth coordinate.

5. The method according to claim 4, characterized in that Determining a fourth coordinate of the third detection frame based on the target prediction rate, the first time, the third coordinate, and the third time includes: The fourth coordinate is determined based on the target predicted rate, the first time, the third coordinate and the third time according to the following formula: Px=(t3-t2)*Vx+X, Py=(t3-t2)*Vy+Y, Among them, (X, Y) represents the third coordinate, (Px, Py) represents the fourth coordinate, and t3 represents the third time.

6. The method according to claim 1, characterized in that The method further comprises: When it is determined that first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is greater than or equal to the predetermined threshold, the number of records included in the first record information is reset to a predetermined value, and the target rate parameter included in the first record information is updated to the target predicted rate.

7. The method according to claim 6, characterized in that The method further comprises: When it is determined that the target state information of the target object is in a predetermined state, the first record information is deleted, wherein the target information includes the target state information of the target object, and the target state information is used to indicate the appearance, disappearance or update of the target object.

8. The method according to claim 1, characterized in that After determining third detection frame information of the target object in a third frame image based on the target prediction rate, the method further includes: The third detection frame information is sent to a target display device to instruct the target display device to determine a third detection frame based on the third detection frame information, superimpose the third detection frame on the third frame image, and display the superimposed image.

9. A device for determining detection frame information, characterized in that: include: An acquisition module, used for acquiring a first frame image, wherein the first frame image is a frame image included in a target image frame group obtained after a predetermined processing is performed on a target video stream; an analysis module, configured to analyze the first frame image to obtain an analysis result, wherein the analysis result includes target information of the target object, the target information includes a target identifier of a first detection frame of the target object, a first coordinate of the first detection frame, and a first time, wherein the first time is used to indicate a time when the first frame image is acquired; a first determination module, configured to determine a target prediction rate of the target object based on the first coordinate, the first time, and second target information when it is determined that the target identifier exists in the target storage unit and the number of records of the target identifier in the target storage unit is greater than or equal to a predetermined threshold, wherein the second target information is used to indicate information including the target object obtained by analyzing a second frame image, the second frame image is an image captured before the first frame image, and the target prediction rate is used to indicate a predicted moving rate of the target object after the first time; a second determination module, configured to determine third detection frame information of the target object in a third frame image based on the target prediction rate, wherein the third frame image is an image acquired after the first frame image; The device is further configured to: if it is determined that the target identifier does not exist in the target storage unit, create first record information in the target storage unit, wherein the first record information includes the target identifier, the number of records and a target rate parameter, set the number of records to a predetermined value, set the target rate parameter to 0, and use the first detection frame to determine the third detection frame information; The device is also used to: when it is determined that the first record information has been established in the target storage unit and the first record information includes the target identifier, and the number of records included in the first record information is less than the predetermined threshold, perform an add 1 operation on the number of records included in the first record information, and determine the third detection frame information based on the target rate parameter recorded in the first record information.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 8 when executed by a processor.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images