An abnormal state detection method and device of a target and an electronic device

Abstract

Embodiments of the present application provide a target abnormal state detection method and device and electronic equipment, and relate to the technical field of intelligent urban management, which can detect whether a barrier appears in an abnormal state. In the method, the position reference information of a first target and the position reference information of a second target in a preset area in a to-be-processed image are acquired; the distance between the first target and the second target is determined according to the position reference information of the first target and the position reference information of the second target; and if the distance is greater than or equal to a first threshold value, it is determined that target loss occurs between the first target and the second target, wherein the first threshold value is determined according to the size information of the first target and the second target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart urban management, and particularly relates to a target abnormal state detection method and device and electronic equipment. BACKGROUND

[0002] In recent years, in order to facilitate citizens to drive, governments in various places are continuously constructing urban expressways connecting different hotspots in urban areas. In the process of constructing urban expressways, in order to avoid citizens driving into a branch road section that has not been completely constructed, a continuous movable barrier is usually temporarily set at a road intersection to remind vehicles to detour. When the road is completely constructed, the barrier can be conveniently removed. In addition, vehicles driving on the urban expressway have a high speed, and when a vehicle suddenly loses control or has an accident on the urban expressway, the flexible barrier can to some extent buffer to reduce the harm to personnel and vehicles.

[0003] However, in daily life, occasionally some citizens move the barrier at will when no one is on duty, so that a gap appears in the continuous barrier to facilitate passing. This brings great hidden dangers to road construction safety and quality and traffic safety. Therefore, how to timely detect whether the barrier is in an abnormal state needs to be solved. SUMMARY

[0004] Embodiments of the present application provide a target abnormal state detection method and device and electronic equipment to timely detect whether the barrier is in an abnormal state.

[0005] In a first aspect, the embodiments of the present application provide a target abnormal state detection method, comprising: acquiring position reference information of a first target and position reference information of a second target in a preset region in a to-be-processed image; determining a distance between the first target and the second target according to the position reference information of the first target and the position reference information of the second target; and determining that target loss occurs between the first target and the second target if the distance is greater than or equal to a first threshold, wherein the first threshold is determined according to size information of the first target and the second target.

[0006] Based on the above scheme, by analyzing and judging the distance between each two adjacent targets, it is determined whether target loss or displacement occurs between the targets of the to-be-processed image. The hidden dangers of barrier loss or displacement to road construction safety and quality and traffic safety can be avoided.

[0007] In a possible implementation, the first target and the second target are adjacent candidate targets in a plurality of targets in a preset region in the image to be processed, or the first target and the second target are candidate targets that are separated by a preset number of targets in the plurality of targets in the preset region in the image to be processed.

[0008] In a possible implementation, the image to be processed is a current video frame in a video in which a target is photographed, and the method further includes: in response to the distance being greater than or equal to the first threshold, increasing a value of a timing flag by a reference value, the timing flag being used to indicate a number of video frames in which a target loss exists in the video in which the target is photographed; and determining whether the value of the timing flag is greater than a second threshold, and in response to the value of the timing flag being greater than the second threshold, sending first alarm information, the first alarm information being used to indicate that a target loss exists in the video in which the target is photographed.

[0009] Based on the foregoing scheme, sending the first alarm information when the timing flag is greater than the second threshold can reduce false positives of target loss caused by short-time vehicle or pedestrian occlusion.

[0010] In a possible implementation, in response to the distance being less than the first threshold, the value of the timing flag is set to a base value.

[0011] Based on the foregoing scheme, when the distance between the first target and the second target is less than the first threshold, it can be determined that no target loss exists in the image to be processed, and thus the timing flag is set to the base value, thereby avoiding false positives of target loss caused by short-time vehicle or pedestrian occlusion.

[0012] In a possible implementation, in response to the value of the timing flag being less than or equal to the second threshold, a next video frame of the current video frame in the video in which the target is photographed is obtained as the image to be processed.

[0013] Based on the foregoing scheme, when the timing flag is less than or equal to the second threshold, the next video frame is obtained, that is, when it is not possible to accurately determine target loss, the first alarm information is not sent. Thus, false positives of target loss caused by short-time vehicle or pedestrian occlusion can be reduced.

[0014] In a possible implementation, the position reference information includes a key point of a corresponding target, and the key point is determined based on a detection box of the corresponding target in the image to be processed.

[0015] Based on the foregoing scheme, because the arrangement of the isolation piers is not on the same horizontal line, the key point of the target can be determined according to the actual situation, to determine the distance between the targets, and thus the accuracy of determining whether target loss exists in the image to be processed can be improved.

[0016] In a possible implementation, the key points include a center point of the corresponding target, and the obtaining of the position reference information of the first target and the position reference information of the second target in the preset region in the to-be-processed image includes: determining a first detection frame of the first target and a second detection frame of the second target, the first detection frame and the second detection frame being obtained by performing target detection on the to-be-processed image by using a target detection network; determining a center point of the first detection frame as the position reference information of the first target; and determining a center point of the second detection frame as the position reference information of the second target.

[0017] Based on the above scheme, the detection frames of each target in the to-be-processed image can be detected by using the target detection network, and the target in the preset region can be determined by using the center point coordinates, so that the interference caused by the target outside the preset region can be avoided, and the accuracy of target detection is improved.

[0018] In a possible implementation, the region image of each target in the preset region is determined from the to-be-processed image according to the detection frame of each target in the preset region in the to-be-processed image; and the region image of each target in the preset region is input into a target classification network respectively, to determine whether the state of each target is abnormal.

[0019] Based on the above scheme, the abnormal state of a single isolation pier can be identified by using the target classification network. The hidden dangers caused by damage or breakage of the isolation pier to road construction safety and quality and traffic safety can be avoided.

[0020] In a possible implementation, the inputting of the region image of each target in the preset region into the target classification network to determine whether the state of each target is abnormal includes: in response to the target in the each target being in a preset state, sending second alarm information for the target in the preset state.

[0021] Based on the above scheme, the abnormal state of a single isolation pier can be alarmed by using the target classification network. The hidden dangers caused by damage or breakage of the isolation pier to road construction safety and quality and traffic safety can be avoided.

[0022] In a possible implementation, for any one target in a preset region in the image to be processed, a common area ratio of the any one target and each target in a previous frame image is determined according to a detection box of the any one target and detection boxes of the targets in the previous frame image, the previous frame image is a previous video frame of the image to be processed in a video in which the image to be processed is located, and the common area ratio is used to represent a ratio of an intersection area of two detection boxes to a union area of the two detection boxes; if the previous frame image does not include a target with the common area ratio with the any one target greater than or equal to a third threshold, it is determined that the previous frame image does not include the any one target, a new target identifier is created for the any one target, and location reference information of the any one target in the image to be processed is stored; if the previous frame image includes a target with the common area ratio with the any one target greater than or equal to the third threshold, it is determined that the previous frame image includes the any one target, and the location reference information of the any one target in the image to be processed is updated.

[0023] Based on the foregoing scheme, by matching the target in the image to be processed and each target in the previous frame image according to the common area ratio, the location reference information of the target already existing in the previous frame image can be updated, and the target not existing in the previous frame image can be added. Therefore, the location reference information of each target can be more accurately determined to determine whether the target loss condition exists in the image to be processed.

[0024] In a second aspect, an embodiment of the present application provides a target abnormal state detection device, comprising:

[0025] An acquisition unit is configured to acquire location reference information of a first target and location reference information of a second target in a preset region in an image to be processed.

[0026] A processing unit is configured to determine a distance between the first target and the second target according to the location reference information of the first target and the location reference information of the second target, and determine that target loss occurs between the first target and the second target if the distance is greater than or equal to a first threshold, wherein the first threshold is determined according to size information of the first target and the second target.

[0027] In a possible implementation, the first target and the second target are adjacent candidate targets in a plurality of targets in the preset region in the image to be processed, or the first target and the second target are candidate targets of targets spaced by a preset number of targets in the plurality of targets in the preset region in the image to be processed.

[0028] In a possible implementation, the image to be processed is a current video frame in a video in which the target is photographed, and the processing unit is further configured to: in response to the distance being greater than or equal to the first threshold, increasing a value of a timing flag by a reference value, the timing flag being used to represent a number of video frames in which the target is lost in the video in which the target is photographed; and determining whether the value of the timing flag is greater than a second threshold, and in response to the value of the timing flag being greater than the second threshold, sending first alarm information, the first alarm information being used to indicate that the target is lost in the video in which the target is photographed.

[0029] In a possible implementation, the processing unit is further configured to: in response to the distance being less than the first threshold, setting the value of the timing flag to a base value.

[0030] In a possible implementation, the acquisition unit is further configured to: in response to the value of the timing flag being less than or equal to the second threshold, acquiring a next video frame of the current video frame in the video in which the target is photographed as the image to be processed.

[0031] In a possible implementation, the position reference information includes a key point of the corresponding target, and the key point is determined based on a detection box of the corresponding target in the image to be processed.

[0032] In a possible implementation, the key point includes a center point of the corresponding target. Before the acquisition unit acquires the position reference information of the first target and the position reference information of the second target in the preset area in the image to be processed, the processing unit is further configured to: determine a first detection box of the first target and a second detection box of the second target, the first detection box and the second detection box being obtained by performing target detection on the image to be processed by using a target detection network; determine a center point of the first detection box as the position reference information of the first target; and determine a center point of the second detection box as the position reference information of the second target.

[0033] In a possible implementation, the acquisition unit is further configured to: determine, from the image to be processed, a region image of each target in a preset area in the image to be processed according to a detection box of each target in the preset area in the image to be processed; and input the region image of each target in the preset area into a target classification network respectively, to determine whether a state of each target is abnormal.

[0034] In a possible implementation, when the processing unit inputs the region image of each target in the preset area into the target classification network respectively, to determine whether the state of each target is abnormal, the processing unit is further configured to: in response to the each target including a target in a preset state, sending second alarm information for the target in the preset state.

[0035] In a possible implementation, the processing unit is further configured to: for any one target in a preset region in the image to be processed, determine a common area ratio of the any one target and each target in a previous frame image according to a detection frame of the any one target and detection frames of the targets in the previous frame image, the previous frame image being a previous video frame of the image to be processed in a video in which the image to be processed is located, the common area ratio representing a ratio of an intersection area of the detection frames of the two targets to a union area of the detection frames of the two targets; if the previous frame image does not include a target with the common area ratio with the any one target greater than or equal to a third threshold, it is determined that the previous frame image does not include the any one target, a new target identifier is created for the any one target, and location reference information of the any one target in the image to be processed is stored; if the previous frame image includes a target with the common area ratio with the any one target greater than or equal to the third threshold, it is determined that the previous frame image includes the any one target, and the location reference information of the any one target in the image to be processed is updated.

[0036] In a third aspect, an electronic device is provided, including:

[0037] a memory configured to store computer instructions;

[0038] a processor connected to the memory, configured to execute the computer instructions in the memory, and implement the method in any one of the first aspect when executing the computer instructions.

[0039] In a fourth aspect, a computer readable storage medium is provided, including:

[0040] The computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer instructions make the computer execute the method in any one of the first aspect.

[0041] The technical effects of each aspect in the second aspect to the fourth aspect and each aspect that can be achieved are refer to the technical effect descriptions of the first aspect or the various possible schemes in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application.

[0043] Figure 1An application scenario of a target abnormal state detection method provided by an embodiment of the present application is shown in the figure.

[0044] Figure 2 An exemplary flow chart of a target abnormal state detection method provided by an embodiment of the present application is shown in the figure.

[0045] Figure 3 A preset area diagram provided by an embodiment of the present application is shown in the figure.

[0046] Figure 4 A structure diagram of a target abnormal state detection system provided by an embodiment of the present application is shown in the figure.

[0047] Figure 5 A target detection module flow chart provided by an embodiment of the present application is shown in the figure.

[0048] Figure 6 A flow chart of an alarm logic judgment module provided by an embodiment of the present application is shown in the figure.

[0049] Figure 7 A target abnormal state detection device diagram provided by an embodiment of the present application is shown in the figure.

[0050] Figure 8 A structure diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments described in the present application document, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the technical solutions of the present application.

[0052] The terms "first" and "second" in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprises" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units that are not listed, or can optionally include other steps or units inherent to the process, method, product or device. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0053] In addition, the term "and / or" in this document merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B can represent three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects without special description.

[0054] In recent years, in order to facilitate citizens to drive, cities have been building urban expressways to connect different hotspots in urban areas. In the process of building urban expressways, in order to avoid citizens driving into the branch road section that has not been completely built, a continuous movable barrier is usually temporarily set at the intersection of the road to warn vehicles to detour. In addition, the vehicle speed on the urban expressway is relatively fast, and when the vehicle suddenly loses control or has an accident on the urban expressway, the flexible barrier can to some extent buffer to reduce the harm to personnel and vehicles. However, in daily life, occasionally some citizens will move the continuous barrier to make a gap for passing through when no one is watching, which will bring great hidden dangers to road construction safety and quality, and traffic safety. Therefore, how to timely identify the abnormal state of the displacement loss or damage, and the collapse of the barrier is urgent to be solved.

[0055] Therefore, the embodiment of the present application provides a target abnormal state detection method. In this method, the position reference information of the first target and the second target in the preset area in the to-be-processed image can be determined by detecting the first target and the second target. And whether the target loss occurs in the to-be-processed image is determined according to the distance between the first target and the second target. When this method is applied to detect the displacement loss of the barrier, the hidden dangers to road construction and traffic safety caused by the loss of the barrier can be avoided.

[0056] Figure 1 The application scenario of the target abnormal state detection method provided by the embodiment of the present application is shown in the figure, which includes a collection device and a computer device. The collection device can be various types of cameras arranged near a road including one or more barriers, used to collect videos of one or more barriers, and send the collected videos to the computer device. The computer device can be used to store the videos sent by the collection device, and can analyze and process the videos, such as analyzing the position reference information of one or more barriers in the preset area in the to-be-processed image in the video, and whether the barrier loss occurs in the to-be-processed image.

[0057] In practice, the acquisition device and the computer device can be connected through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a wireless fidelity (WIFI) network. Of course, other possible networks can also be used, and the application does not make any limitation in this regard.

[0058] After introducing the exemplary application scenarios of the embodiments of the application, the technical solutions provided by the embodiments of the application will be described in detail below in combination with the drawings and specific implementation manners. Although the embodiments of the application provide the method operation steps as described in the following embodiments or shown in the drawings, more or fewer operation steps can be included in the method based on conventional or non-creative labor. The execution order of the steps is not limited to the execution order provided by the embodiments of the application in logical terms. In the actual processing process or when the control device is executed, the method can be executed in sequence or in parallel as shown in the embodiments or the drawings.

[0059] Referring to Figure 2 An exemplary flowchart of a target abnormal state detection method provided by the embodiments of the application can be applied to a computer device as shown in Figure 1 The method can include the following flow:

[0060] S201, obtaining position reference information of a first target and position reference information of a second target in a preset region in a to-be-processed image.

[0061] The to-be-processed image can be a current video frame in a video of the target captured by the acquisition device in real time as shown in Figure 1 Optionally, when the target state detection method provided by the embodiments of the application is used to detect the state of the target in an offline video, the to-be-processed image can be any video frame in the video of the target.

[0062] In some embodiments, the preset region can be a closed region set in advance according to the possible position of the target in each video frame of the video of the target. For example, because the position of the acquisition device is fixed, the position of the isolation mound is also fixed within a certain period of time, and therefore the position of the isolation mound in each image is also relatively fixed. Referring to Figure 3 A schematic diagram of the preset region provided by the embodiments of the application is shown. Assuming that the to-be-processed image is as shown in Figure 3 The region in the dashed box in the to-be-processed image is the preset region, and the solid line box in the preset region is a single isolation mound.

[0063] In a possible scenario, the preset region in the image to be processed can include multiple targets, and the first target and the second target can be adjacent candidate targets in the multiple targets in the preset region in the image to be processed. For example, it is assumed that the preset region in the image to be processed includes a roadblock A, a roadblock B, and a roadblock C. The roadblock A is adjacent to the roadblock B, the roadblock B is adjacent to the roadblock A and the roadblock C, and the roadblock C is adjacent to the roadblock B. Therefore, when the roadblock A is the first target, the roadblock B can be the second target. When the roadblock B is the first target, the roadblock A and the roadblock C can be the second target. When the roadblock C is the first target, the roadblock B can be the second target.

[0064] In another possible scenario, the first target and the second target can also be candidate targets spaced apart by a preset number of targets in the multiple targets in the preset region in the image to be processed. For example, it is assumed that the preset region in the image to be processed includes a roadblock A, a roadblock B, a roadblock C, a roadblock D, and a roadblock E. The five roadblocks are arranged in the order of the roadblock A, the roadblock B, the roadblock C, the roadblock D, and the roadblock E. When the preset number is 1, the roadblock A is the first target, and the roadblock C can be the second target. If the roadblock B is the first target, the roadblock D can be the second target. If the roadblock C is the first target, the roadblock A and the roadblock E can be the second target, and so on.

[0065] In some embodiments, the position reference information includes a key point of the corresponding target, where the key point can be determined based on a detection frame of the corresponding target in the image to be processed.

[0066] In a possible implementation, the key point of the corresponding target included in the position reference information can be a vertex of a detection frame of the corresponding target. For example, the key point of each target in the preset region in the image to be processed can be a top-left vertex of a detection frame of each target. The key point of the corresponding target included in the position reference information can also be a midpoint of a side of a detection frame of the corresponding target. For example, the key point of each target in the preset region in the image to be processed can be a midpoint of a short side on which a top-left vertex of a detection frame of each target is located.

[0067] The detection frame of each target in the preset region in the image to be processed can be obtained by the following method: the computer device can input the image to be processed collected by the acquisition device into a target detection network. Each target in the image to be processed can be detected by the target detection network to obtain the detection frame of each target, the coordinates (x0, y0) of the top-left vertex of the detection frame, and the coordinates (x1, y1) of the bottom-right vertex of the detection frame. That is, the first detection frame of the first target and the second detection frame of the second target in the preset region in the image to be processed, and the coordinates of the top-left vertex and the bottom-right vertex of the first detection frame and the coordinates of the top-left vertex and the bottom-right vertex of the second detection frame can be determined by the above method.

[0068] It should be appreciated that the target detection network can be any one of a network or any one of a derivative improved network based on image or video target detection or key point detection. For example, the target detection network can be a yolov3 detection network, which is not limited in the present application.

[0069] In a possible implementation, the key point of the corresponding target included in the position reference information can also be the center point of the detection frame of the corresponding target. The center point coordinates of the detection frame of each target in the image to be processed can be determined according to the coordinates of the top-left vertex and the bottom-right vertex of the detection frame, and can satisfy formula (1). Then the center point coordinates of the detection frame of the target in the preset region in the image to be processed can be determined.

[0070]

[0071] In the formula, x center is the horizontal coordinate of the center point coordinates, and y center is the vertical coordinate of the center point coordinates.

[0072] In a possible implementation, the training sample set used in the training process of the above target detection network can be obtained by collecting video frame images in videos including targets in various scenes. And the targets in normal state and the targets in abnormal state in the video frame images can be trained as positive samples in the training sample set, so that the target detection network can detect all the targets in the video frame images. The abnormal state can include states such as lodging and damage. Various scenes can include different weather scenes such as overcast, sunny, rainy, etc., and can also include different environment scenes such as expressway, urban trunk road, etc. The present application does not limit the scene.

[0073] Based on the above scheme, the detection frame of each target in the image to be processed can be detected by the target detection network, and the target in the preset region can be determined by the center point coordinates, which can avoid the interference of the target outside the preset region, thereby improving the accuracy of target detection.

[0074] S202, determine the distance between the first target and the second target according to the position reference information of the first target and the position reference information of the second target.

[0075] The computer device can determine the distance between the first target and the second target according to the distance between the key points of the first detection frame and the key points of the second detection frame.

[0076] In one example, the first target and the second target are adjacent candidate targets in a plurality of targets in a preset region in the to-be-processed image, and the key points are the center points. For the plurality of targets in the preset region of the to-be-processed image, determining the distance between the first target and the second target can be determining the distance between every two adjacent candidate targets in the preset region in the to-be-processed image. Therefore, the distance between every two adjacent candidate targets can be determined in turn according to the horizontal coordinate values of the center point coordinates of each target in an increasing manner. For example, the preset region of the to-be-processed image can include a barrier A, a barrier B and a barrier C. The horizontal coordinate values of the center point coordinates of the barrier A, the barrier B and the barrier C are in turn increasing. Therefore, the distance between the barrier A and the barrier B can be determined first, and then the distance between the barrier B and the barrier C can be determined.

[0077] S203, if the distance is greater than or equal to a first threshold, it is determined that target loss occurs between the first target and the second target.

[0078] The first threshold is determined according to the size information of the first target and the second target. For example, it can be determined according to the width of the first target and the second target.

[0079] In one example, the first threshold can be half of the sum of the width of the first target and the width of the second target, satisfying formula (2).

[0080]

[0081] In the formula, W is the first threshold, x0 is the horizontal coordinate of the top left vertex of the detection frame of the first target, x1 is the horizontal coordinate of the bottom right vertex of the detection frame of the first target, then |x0-x1| is the width of the first target. x'0 is the horizontal coordinate of the top left vertex of the detection frame of the second target, x'1 is the horizontal coordinate of the bottom right vertex of the detection frame of the second target, then |x'0-x'1'| is the width of the second target.

[0082] In another example, an error can also be allowed when calculating the first threshold according to the widths of the first target and the second target. Wherein, the maximum allowed error can be δ, and the first threshold can satisfy formula (3). It should be appreciated that the maximum allowed error can be set according to actual conditions or experience, such as 10% or 20%, and the present application does not limit this.

[0083]

[0084] In a possible implementation, if the distance between the first target and the second target is greater than or equal to the first threshold, the computer device can send first alarm information to the terminal device of the relevant worker or the computer device for managing the target state. Wherein, the first alarm information is used to indicate that there is a target loss in the video of the target.

[0085] In another possible implementation, if the distance between the first target and the second target is less than the first threshold, the next frame of the current video frame in the video of the target can be obtained as the to-be-processed image, and the processes of S201-S203 are repeatedly executed.

[0086] Based on the above scheme, by analyzing and judging the distance between each adjacent two targets, it is determined whether the target loss or displacement occurs between the targets of the to-be-processed image. The hidden danger caused by the isolation pier loss or displacement to the road construction safety and quality, and traffic safety can be avoided.

[0087] In some embodiments, in order to reduce the false alarm of target loss caused by short-time vehicle or pedestrian shielding, the computer device can also set a timing flag to determine the frame number of target loss. When the distance between the first target and the second target in the to-be-processed image is greater than or equal to the first threshold, it is determined that there can be a target loss between the first target and the second target, the value of the timing flag can be increased by a reference value, and whether the value of the timing flag is greater than a second threshold is judged.

[0088] If the value of the timing flag is greater than the second threshold, the computer device can send first alarm information to the terminal device of the relevant worker or the computer device for managing the target state. If the value of the timing flag is less than or equal to the second threshold, the next frame of the current video frame in the video of the target can be obtained as the to-be-processed image.

[0089] If the distance between the first target and the second target in the to-be-processed image is less than the first threshold, the computer device can set the value of the timing flag to a basic value. It should be appreciated that the reference value, the basic value and the second threshold can be set according to experience or actual conditions, for example, the reference value can be 1, the basic value can be 0, and the second threshold can be 12, and the present application does not limit this.

[0090] For example, it is assumed that the barrier A and the barrier B can be detected in the image to be processed, and the value of the timing mark is 12, the second threshold is 12, and the base value is 0. If the distance between the barrier A and the barrier B is greater than or equal to the first threshold at this time, the value of the timing mark is increased by 1, and the value of the timing mark is 13 at this time. Since the value of the timing mark at this time is greater than the second threshold, the computer device can send the first alarm information to the relevant staff. If the distance between the barrier A and the barrier B is less than the first threshold at this time, the value of the timing mark is set to 0. Since the value of the timing mark at this time is less than the second threshold, the computer device can obtain the next frame of the current video frame in the video in which the target is photographed as the image to be processed.

[0091] In some embodiments, after the computer device determines the detection frame of each target in the preset area in the image to be processed by the target detection network, the computer device can also determine the region image of each target in the preset area from the image to be processed according to the detection frame of each target. And input the region image of each target into the target classification network respectively, to determine whether the state of each target is abnormal. Wherein, the abnormal state can include lodging or damage. The region image of the target can be the image in the region of the detection frame of the target.

[0092] In some embodiments, if it is determined that the target in the preset area in the image to be processed includes a target with an abnormal state, the computer device can send the second alarm information to the terminal device of the relevant staff or the computer device for managing the state of the target in response to the target in the preset area in the image to be processed including the target with the preset state. Wherein, the preset state can be lodging and / or damage. For example, when the preset state is lodging, the computer device can send the second alarm information when the target in each target includes the target with the state of lodging. When the preset state is damage, the computer device can send the second alarm information when the target in each target includes the target with the state of damage. When the preset state is lodging and damage, the computer device can send the second alarm information when the target in each target includes the target with the state of lodging or damage.

[0093] Optionally, for any one target in the preset area in the image to be processed, when determining the state of the target by the target classification network, the confidence of the target in each state can be output. And the state with the highest confidence is selected as the state of the target. Wherein, the confidence is a floating point number in the range of 0 to 1. For example, when identifying the state of the barrier A by the target classification network, the confidence of the state of the barrier A as normal is 0.8, the confidence of the state of the barrier A as damage is 0.1, and the confidence of the state of the barrier A as lodging is 0.2. Since the confidence of the state of the barrier A as normal is the largest, the state of the barrier A can be determined as normal.

[0094] In one example, since the isolation mound includes the following two features: 1) the color of the isolation body is usually red and yellow, and yellow is the superposition of red and green in the RGB color system. 2) through Figure 3 As can be seen, the upper half of the isolation body of the isolation mound can identify several hole regions, while the fallen isolation mound cannot identify several hole regions. Based on the above features of the isolation mound, the attention mechanism can be used to make the target classification network pay more attention to the parts with larger differences. Therefore, the target classification network can use a classification model combining RGB channel attention and image region attention.

[0095] Based on the above scheme, through investigation and analysis of the isolation mound, the embodiment of the present application provides a classification network combining color channel attention and image spatial attention for the color features and shape features of the isolation body of the isolation mound. It can not only comprehensively identify all isolation mounds in the current frame, but also accurately classify the state of each isolation mound target. Thus, the accuracy of isolation mound detection and the accuracy of isolation mound state recognition can be provided.

[0096] It should be understood that the above target classification network can also use any network or any derivative improved network based on image recognition or classification. The present application does not limit this.

[0097] In some embodiments, the training sample set used in the training process of the target classification network can be obtained by collecting video frame images in videos including targets in various scenes. And the normal state target in the video frame image is taken as a positive sample, and the fallen state and damaged state target is taken as a different class sample to perform multi-classification training on the target classification network, so that the target classification network can accurately identify the different states of the target.

[0098] In one possible implementation, since the number of samples of the normal state target is greater than the number of samples of the abnormal state target in the sampling process. Therefore, the number of samples of the abnormal state target can be increased by adding noise, rotation transformation and other processing methods to the samples of the abnormal state target. So that the order of magnitude of the number of samples of the normal state target and the order of magnitude of the number of samples of the abnormal state target are consistent in the training process of the target detection network and the target classification network, thereby improving the accuracy of the target detection network and the target classification network.

[0099] Based on the same concept of the above method, see Figure 4 A structure diagram of a target abnormal state detection system provided by an embodiment of the present application. The system 400 can include a video acquisition module 401, a target detection module 402, a multi-target tracking module 403, and an alarm logic judgment module 404.

[0100] The video acquisition module 401 is configured to acquire a video of a target and send the acquired video to the target detection module 402. The acquisition method can refer to the related description in the method embodiment as shown in Figure 2 The method embodiment is not repeated here.

[0101] The target detection module 402 can be composed of a target detection network and a target classification network. The target detection module 402 is configured to detect and classify the target in the image to be processed in the video sent by the video acquisition module 401, and send the integrated target information to the multi-target tracking module 403. The target information can include the position reference information of the target and the state of the target.

[0102] The target detection module provided by the embodiment of the present application is described in detail. Figure 5 The flowchart of the target detection module can include the following steps.

[0103] S501, input an image to be processed.

[0104] S502, a target detection network.

[0105] The target detection module 402 can detect the target in the image to be processed through the target detection network. The detection method can refer to the related description in the method embodiment as shown in Figure 2 The method embodiment is not repeated here.

[0106] S503, whether the target is in a preset area.

[0107] If the target detected by the target detection network is a target in the preset area, S504 is executed to intercept the region image of each target in the image to be processed according to the detection frame of each target, and send the region image of each target to the target classification network. If it is not a target in the preset area, no operation is performed. After determining the detection frame of each target through the target detection network, whether the target is in the preset area can be determined through the center point coordinates. The specific method can refer to the related description in the method embodiment as shown in Figure 2 The method embodiment is not repeated here.

[0108] S504, a target classification network.

[0109] The target detection module 402 can identify the region image of each target through the target classification network, so as to determine the state of each target. The determination method of the target state can refer to the related description in the above method embodiment, which is not repeated here.

[0110] S505, integrate target information.

[0111] For any one target, target information can be integrated according to position reference information of the target determined by the target detection network and a state of the target determined by the target classification network. The position reference information includes information of a detection frame.

[0112] The multi-target tracking module 403 is configured to match each target in the to-be-processed image and each target in a previous frame image according to target information of each target in the to-be-processed image determined by the target detection module 402. The previous frame image is a previous video frame of the to-be-processed image in a video in which the to-be-processed image is located.

[0113] In one example, for any one target in the to-be-processed image, a common area ratio (Intersection of Union, IoU) of the target and each target in the previous frame image can be determined according to a detection frame of the target in the to-be-processed image and the detection frames of the targets in the previous frame image, respectively. The IoU is used to represent a ratio of an area of an intersection part of the detection frames of the two targets to an area of a union part of the detection frames of the two targets.

[0114] For example, it is assumed that the to-be-processed image includes a barrier A and a barrier B, and the previous frame image includes a barrier C and a barrier D. For the barrier A, the IoU of the barrier A and the barrier C and the IoU of the barrier A and the barrier D can be determined according to the detection frame of the barrier A, the detection frame of the barrier C, and the detection frame of the barrier D. Similarly, for the barrier B, the IoU of the barrier B and the barrier C and the IoU of the barrier B and the barrier D can be determined according to the detection frame of the barrier B, the detection frame of the barrier C, and the detection frame of the barrier D.

[0115] In another example, for any one target in the to-be-processed image, a detection frame of a target at a similar region of a detection frame of the target in the previous frame image can also be determined according to the detection frame of the target, and the IoU of the two targets can be determined according to the detection frame in the to-be-processed image and the detection frame in the previous frame image. The similar region can refer to a region within a fourth threshold range from the detection frame of the target in the to-be-processed image. It should be noted that the fourth threshold can be preset according to actual conditions or experience, and the present application does not make any limitation thereto.

[0116] In some embodiments, when the multi-target tracking module 403 determines whether each target in the to-be-processed image and each target in the previous frame image match according to the IoU, the following three cases can be included:

[0117] Case one: if the target in the previous frame of image does not include a target with an IoU greater than or equal to the third threshold value, it is determined that the previous frame of image does not include the target, i.e., the target is a new target, a new target identifier is created for the target, and the position reference information of the target is stored.

[0118] Case two: if the target in the previous frame of image includes a target with an IoU greater than or equal to the third threshold value, it is determined that the previous frame of image includes the target, and the position reference information of the target in the image to be processed is updated.

[0119] Case three: if there is a target in the previous frame of image, and the IoU between the target and any target in the image to be processed is less than the third threshold value, it is determined that the target is lost in the image to be processed.

[0120] In one example, the multi-target tracking module can set a state flag for each target. The state flag can include a Create flag, an Update flag, a Lost flag, and a Delete flag. When a target in the image to be processed is as shown in case one described above, the Create flag can be set for the target identifier corresponding to the target. When a target in the image to be processed is as shown in case two described above, the Update flag can be set for the target identifier corresponding to the target. When a target in the previous frame of image is as shown in case three described above, the Lost flag can be set for the target identifier corresponding to the target. When the number of frames in which a target identifier maintains the Lost flag is greater than the second threshold value, the Delete flag can be set for the target identifier, and the target information corresponding to the target identifier is deleted.

[0121] The alarm logic judgment module 404 is configured to issue an alarm information when a target appears in an abnormal state or is lost. Referring to Figure 6 , a flowchart of the alarm logic judgment module provided by the embodiments of the present application can include:

[0122] S601, information of each target is cyclically obtained.

[0123] According to the multi-target tracking module 403, the matching result of each target in the image to be processed is cyclically obtained, and the target information and the state flag of each target are determined according to the matching result.

[0124] S602, it is determined whether the target is in a preset area.

[0125] If the target is in the preset area, S603 is performed; if the target is not in the preset area, S605 is performed. The method of determining whether the target is in the preset area can refer to the related description in the method embodiment as Figure 2 indicated, and will not be described here again.

[0126] S603, determining whether the state of the target is abnormal.

[0127] If the state of the target is normal, S604 is performed; if the state of the target is abnormal, S606 is performed.

[0128] S604, adding the target into a target set.

[0129] The target set is used for storing each target in the image to be processed.

[0130] S605, determining whether all targets are traversed.

[0131] It is determined whether all targets in the image to be processed are added into the target set, if all targets in the image to be processed are traversed, S609 is performed; if all targets in the image to be processed are not traversed, S601 is performed.

[0132] S606, determining whether the state of the target is lodging.

[0133] If it is determined that the state of the target is lodging, S607 is performed; if it is determined that the state of the target is broken, S608 is performed.

[0134] S607, setting the value of an alarm flag to 2.

[0135] When the value of the alarm flag is set to 2, it can be used to indicate that the state of the target is lodging.

[0136] S608, setting the value of the alarm flag to 3.

[0137] When the value of the alarm flag is set to 3, it can be used to indicate that the state of the target is broken.

[0138] S609, determining whether there is target loss.

[0139] If there is target displacement or loss in the image to be processed, S610 is performed; if there is no target loss in the image to be processed, S612 is performed. The method for determining whether there is target loss in the image to be processed can refer to the related description in the method embodiment as shown in the method embodiment, which will not be described here in detail. Figure 2

[0140] S610, setting the value of the alarm flag to 1.

[0141] When the value of the alarm flag is set to 1, it can be used to indicate that there is target loss in the image to be processed.

[0142] S611, t=t+1.

[0143] The t is used to indicate a timing flag.

[0144] ​S612, set the value of the alarm flag to 0.

[0145] When the value of the alarm flag is set to 0, it indicates that the state of the target is normal.

[0146] It should be understood that the values of the alarm flag described above can be set according to actual conditions. For example, when the value of the alarm flag is 7, it indicates that the state of the target is normal. When the value of the alarm flag is 6, it indicates that the state of the target is lodging. When the value of the alarm flag is 5, it indicates that the state of the target is damaged. When the value of the alarm flag is 4, it indicates that there is target loss in the image to be processed. The present application does not make specific limitations on this.

[0147] S613, t = 0.

[0148] S614, determine whether t is greater than a second threshold value.

[0149] If t is greater than the second threshold value, S615 is executed, and if t is less than or equal to the second threshold value, S616 is executed.

[0150] S615, report the alarm information.

[0151] The value of the alarm flag can correspond to the alarm information one by one, and the alarm logic judgment module 404 can send the alarm information corresponding to the value of the alarm flag to the terminal device of the relevant staff or the computer device used for managing the state of the target.

[0152] S616, obtain the next video frame.

[0153] The next video frame of the image to be processed in the video of the target is obtained, and the process of S601-S616 is continued with the next video frame as the image to be processed.

[0154] Based on the same concept of the above method, see Figure 7 A target abnormal state detection device 700 provided by the embodiment of the present application can execute each step in the above method, and details are not described here to avoid repetition. The device 700 includes an acquisition unit 701 and a processing unit 702. In one scenario:

[0155] The acquisition unit 701 is configured to acquire position reference information of a first target and position reference information of a second target in a preset region in an image to be processed.

[0156] The processing unit 702 is configured to determine a distance between the first target and the second target according to the position reference information of the first target and the position reference information of the second target, and determine that target loss occurs between the first target and the second target if the distance is greater than or equal to a first threshold value, wherein the first threshold value is determined according to size information of the first target and the second target.

[0157] In a possible implementation, the first target and the second target are adjacent candidate targets in a plurality of targets in a preset region in the image to be processed, or the first target and the second target are candidate targets that are separated by a preset number of targets in the plurality of targets in the preset region in the image to be processed.

[0158] In a possible implementation, the image to be processed is a current video frame in a video in which a target is photographed, and the processing unit 702 is further configured to: in response to the distance being greater than or equal to the first threshold, increase a value of a timing flag by a reference value, the timing flag being used to indicate a number of video frames in which a target is lost in the video in which the target is photographed; and determine whether the value of the timing flag is greater than a second threshold, and in response to the value of the timing flag being greater than the second threshold, send first alarm information, the first alarm information being used to indicate that a target is lost in the video in which the target is photographed.

[0159] In a possible implementation, the processing unit 702 is further configured to: in response to the distance being less than the first threshold, set the value of the timing flag to a base value.

[0160] In a possible implementation, the acquisition unit 701 is further configured to: in response to the value of the timing flag being less than or equal to the second threshold, acquire a next video frame of the current video frame in the video in which the target is photographed as the image to be processed.

[0161] In a possible implementation, the position reference information includes a key point of a corresponding target, and the key point is determined based on a detection box of the corresponding target in the image to be processed.

[0162] In a possible implementation, the key point includes a center point of the corresponding target. Before the acquisition unit 701 acquires the position reference information of the first target and the position reference information of the second target in a preset region in the image to be processed, the processing unit 702 is further configured to: determine a first detection box of the first target and a second detection box of the second target, the first detection box and the second detection box being obtained by performing target detection on the image to be processed by using a target detection network; determine a center point of the first detection box as the position reference information of the first target; and determine a center point of the second detection box as the position reference information of the second target.

[0163] In a possible implementation, the acquisition unit 701 is further configured to: determine, from the to-be-processed image, a region image of each target in a preset region in the to-be-processed image according to a detection frame of each target in the preset region; and input the region image of each target in the preset region into a target classification network respectively, to determine whether a state of each target is abnormal.

[0164] In a possible implementation, when the processing unit 702 inputs the region image of each target in the preset region into the target classification network respectively to determine whether a state of each target is abnormal, the processing unit 702 is further configured to: in response to a target in the each target having a preset state, send second alarm information for the target having the preset state.

[0165] In a possible implementation, the processing unit 702 is further configured to: for any one target in the preset region in the to-be-processed image, determine a common area proportion of the any one target and each target in a previous frame image according to a detection frame of the any one target and detection frames of the each target in the previous frame image, the previous frame image being a previous video frame of the to-be-processed image in a video in which the to-be-processed image is located, the common area proportion being used to represent a ratio of an intersection area of two detection frames to a union area of the two detection frames; if the previous frame image does not include a target having a common area proportion greater than or equal to a third threshold with the any one target, it is determined that the previous frame image does not include the any one target, a new target identifier is created for the any one target, and location reference information of the any one target in the to-be-processed image is stored; if the previous frame image includes a target having a common area proportion greater than or equal to the third threshold with the any one target, it is determined that the previous frame image includes the any one target, and the location reference information of the any one target in the to-be-processed image is updated.

[0166] Based on the same concept of the above method, refer to Figure 8A structural schematic diagram of an electronic device is provided in the embodiments of the present application. The electronic device includes at least one processor 802, and a memory 801 connected or coupled with the at least one processor 802. In addition, the electronic device can further include a communication interface 803. The electronic device can interact with other devices through the communication interface 803. For example, the communication interface 803 can be a transceiver, a circuit, a bus, a module, a pin, or other types of communication interfaces. When the electronic device is a chip device or a circuit, the communication interface 803 in the electronic device can also be an input and output circuit, which can input (or receive) information and output (or send) information. The processor can be an integrated processor or a microprocessor or an integrated circuit or a logic circuit. The processor can determine the output information according to the input information.

[0167] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information interaction between devices, units or modules. The processor 802 can operate cooperatively with the memory 801 and the communication interface 803. The specific connection medium between the processor 802, the memory 801 and the communication interface 803 is not limited in the present application.

[0168] Optionally, referring to Figure 8 , the processor 802, the memory 801 and the communication interface 803 are connected with each other through a bus 840. The bus 800 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 8 only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0169] In the embodiments of the present application, the memory 801 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 801 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 801 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 801 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store instructions, computer programs and / or data.

[0170] In the embodiments of the present application, the processor 802 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, which can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the target abnormal state detection method disclosed in combination with the embodiments of the present application can be directly embodied by the hardware processor for execution, or executed by a combination of hardware and software modules in the processor.

[0171] By designing and programming the processor 802, the code corresponding to the target abnormal state detection method introduced in the foregoing embodiments can be fixed in the chip, so that the chip can execute the steps of the foregoing target abnormal state detection method during runtime. How to design and program the processor 802 is a technology known to those skilled in the art, which will not be described here.

[0172] Specifically, in the embodiments of the present application, the memory 801 stores instructions executable by the at least one processor 802, and the at least one processor 802 can execute the steps included in the target abnormal state detection method by invoking the instructions or computer programs stored in the memory 801. For example, the processor 802 is configured to acquire the center point coordinates of a plurality of targets in a preset region in a to-be-processed image by using the communication interface 803; the processor 802 is further configured to determine the distance between a first target and a second target according to the center point coordinates of the first target and the center point coordinates of the second target, wherein the first target and the second target are adjacent targets in the plurality of targets.

[0173] Further, the processor 802 is further configured to determine that target loss occurs between the first target and the second target if the distance is greater than or equal to a first threshold, wherein the first threshold is determined according to the widths of the first target and the second target.

[0174] The embodiments of the present application also provide a computer readable storage medium, which stores computer instructions, and the computer instructions make the computer execute the steps of any of the above methods when the computer instructions run on the computer.

[0175] In some possible implementation manners, each aspect of the target abnormal state detection method provided by the present application can also be implemented in the form of a computer program product, which includes program codes, and the program codes are used to make the electronic device execute the steps in any of the methods described above when the computer program product runs on the electronic device.

[0176] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The above-mentioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned storage medium includes ROM, RAM, magnetic disk or optical disk and various media that can store program codes.

[0177] Although the specific embodiments of the present application are described above, those skilled in the art should understand that these are only illustrative, and the protection scope of the present application is defined by the appended claims. Those skilled in the art can make various changes or modifications to the embodiments without departing from the principles and essence of the present application, and these changes and modifications all fall within the protection scope of the present application. Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they understand the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0178] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method of detecting an abnormal state of a target, characterized by, The method comprises: obtaining position reference information of a first target and position reference information of a second target in a preset region in a to-be-processed image; the to-be-processed image is a current video frame in a video in which a target is photographed; the first target and the second target are any two bollards in the preset region; determining a distance between the first target and the second target according to the position reference information of the first target and the position reference information of the second target; in response to the distance being greater than or equal to a first threshold, increasing a value of a timing flag by a reference value; the value of the timing flag is used to represent a number of video frames in which target loss exists in the video in which the target is photographed; in response to the distance being less than the first threshold, setting the value of the timing flag to a base value; the first threshold is determined according to size information of the first target and the second target; determining whether the value of the timing flag is greater than a second threshold; in response to the value of the timing flag being greater than the second threshold, sending first alarm information; the first alarm information is used to indicate that target loss exists in the video in which the target is photographed; in response to the value of the timing flag being less than or equal to the second threshold, obtaining a next video frame of the current video frame in the video in which the target is photographed as the to-be-processed image.

2. The method of claim 1, wherein, The first target and the second target are adjacent candidate targets in a plurality of targets in the preset region in the to-be-processed image, or the first target and the second target are candidate targets that are separated by a preset number of targets in the plurality of targets in the preset region in the to-be-processed image.

3. The method of claim 1, wherein, The position reference information comprises a key point of a corresponding target, and the key point is determined based on a detection box of the corresponding target in the to-be-processed image.

4. The method of claim 3, wherein, The key point comprises a center point of the corresponding target, and the method for obtaining the position reference information of the first target and the position reference information of the second target in the preset region in the to-be-processed image comprises: determining a first detection box of the first target and a second detection box of the second target, the first detection box and the second detection box being obtained by performing target detection on the to-be-processed image by a target detection network; determining a center point of the first detection box as the position reference information of the first target; and determining a center point of the second detection box as the position reference information of the second target.

5. The method of claim 3, wherein, The method further comprises: determining a region image of each target in the preset region from the to-be-processed image according to a detection box of each target in the preset region in the to-be-processed image; inputting the region image of each target in the preset region into a target classification network respectively to determine whether a state of the each target is abnormal.

6. The method of claim 5, wherein, The method further comprises: in response to the each target comprising a target in a preset state, sending second alarm information for the target in the preset state.

7. The method according to any one of claims 3-6, characterized in that, The method further comprises: For any one target in a preset region in the to-be-processed image, a common area proportion of the any one target and each target in a previous frame image is determined according to a detection box of the any one target and detection boxes of the targets in the previous frame image; the previous frame image is a previous video frame of the to-be-processed image in a video in which the to-be-processed image is located, and the common area proportion is used to represent a ratio of an intersection area of two detection boxes to a union area of the two detection boxes; If the previous frame image does not include a target whose common area proportion with the any one target is greater than or equal to a third threshold, it is determined that the previous frame image does not include the any one target, a new target identifier is created for the any one target, and position reference information of the any one target in the to-be-processed image is stored; If the previous frame image includes a target whose common area proportion with the any one target is greater than or equal to the third threshold, it is determined that the previous frame image includes the any one target, and the position reference information of the any one target in the to-be-processed image is updated.

8. An abnormal state detection device of an object, characterized by comprising: Comprise: An acquisition unit is configured to acquire position reference information of a first target and position reference information of a second target in a preset region in a to-be-processed image; The to-be-processed image is a current video frame in a video in which a target is photographed; the first target and the second target are any two road barriers in the preset region; A processing unit is configured to determine a distance between the first target and the second target according to the position reference information of the first target and the position reference information of the second target; in response to the distance being greater than or equal to a first threshold, a value of a timing flag is increased by a reference value; the value of the timing flag is used to represent a number of video frames in which a target is lost in the video in which the target is photographed; in response to the distance being less than the first threshold, the value of the timing flag is set to a base value; The first threshold is determined according to size information of the first target and the second target; it is determined whether the value of the timing flag is greater than a second threshold; in response to the value of the timing flag being greater than the second threshold, first alarm information is sent; The first alarm information is used to indicate that a target is lost in the video in which the target is photographed; in response to the value of the timing flag being less than or equal to the second threshold, a next video frame of the current video frame in the video in which the target is photographed is acquired as the to-be-processed image.

9. An electronic device, comprising: Comprise: A memory is configured to store computer instructions; A processor is connected with the memory and is configured to execute the computer instructions in the memory, and when the computer instructions are executed, the method in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, Comprise: The computer readable storage medium stores computer instructions, when the computer instructions run on a computer, make the computer execute the method in any one of claims 1 to 7.

Images