High-altitude falling object tracing system based on artificial intelligence

Abstract

The invention relates to the technical field of image data processing, and discloses a high-altitude falling object tracing system based on artificial intelligence, and the system comprises an image collection module which is used for collecting image data of a target object; the preprocessing module is used for preprocessing the image data to obtain a first target image; the target detection module is used for constructing a target detection model according to the first target image, and the output of the target detection model is a first feature sequence; and the track generation module is used for generating a motion track of the target object according to the first feature sequence in combination with a tracking algorithm. The position of the center point of the bounding box of the target object in each frame of the image is obtained by constructing and training the target detection model, the position of the target object is updated in combination with a tracking algorithm, and the accuracy of the source position of the target object is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image data processing, and in particular relates to a high-altitude falling object tracing system based on artificial intelligence. Background Art

[0002] Accidents involving falling objects from high altitudes occur frequently in urban high-rise buildings, seriously threatening the safety of life and property of pedestrians. With the acceleration of urbanization, there are more and more high-rise buildings, and the number of injuries caused by objects thrown or dropped from high altitudes is increasing year by year. Such accidents not only cause direct casualties and property losses, but may also lead to legal disputes and social conflicts, and have become a major problem in urban safety management.

[0003] At present, the tracing methods for falling objects from high altitudes mainly rely on eyewitness testimony and on-site surveillance videos. These methods can help determine the source of falling objects to a certain extent, but they generally have problems such as low efficiency, poor accuracy, and difficulty in obtaining evidence. Eyewitness testimony is often unreliable due to limited viewing angles or vague memories, and tracing based on surveillance videos is time-consuming and laborious, and is prone to inaccurate results due to poor video quality or missing information.

[0004] Therefore, a high-altitude falling object tracing system is urgently needed to solve the above problems. Summary of the invention

[0005] The present invention provides an artificial intelligence-based high-altitude falling object tracing system to solve the technical problems of low efficiency, poor accuracy, difficulty in evidence collection and high labor cost in related technologies.

[0006] The present invention provides a high-altitude falling object tracing system based on artificial intelligence, comprising:

[0007] An image acquisition module, which is used to acquire image data of a target object;

[0008] The image data includes video data from the appearance of the target object to the disappearance or landing of the target object;

[0009] A preprocessing module, which is used to preprocess the image data to obtain a first target image, wherein the first target image includes N consecutive video frames, where N is a custom parameter;

[0010] A target detection module is used to build a target detection model according to the first target image, wherein the input of the target detection model is the first target image, and the output is a first feature sequence, wherein the nth sequence unit of the first feature sequence represents the target object category, bounding box width, bounding box height, bounding box center point position and confidence of the nth video frame, wherein 1≤n≤N;

[0011] The trajectory generation module is used to generate the motion trajectory of the target object according to the first feature sequence in combination with the tracking algorithm.

[0012] Furthermore, the pre-processing step includes:

[0013] Step S201, by Perform noise removal on the image data, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and Gx, y represents the Gaussian function value at x, y, that is, the weight of the point;

[0014] Step S202, by The image data is enhanced, where Hu, v represents the frequency response of the filter, Du, v represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter;

[0015] Step S203, by Separate the target object and select it in the video frame, where I t x, y represent the video frame image, x, y are spatial coordinates, representing the pixel position in the image, Bx, y represent the background model, I t x,y-Bx,y represents the pixel difference between each frame image and the background model, T represents the first threshold, Mx,y represents the target mask, and the target mask consists of 1 and 0;

[0016] Step S204, arranging the image data in reverse chronological order;

[0017] Step S205: normalize the image data using a z-score normalization method.

[0018] Furthermore, the target detection model is constructed based on a convolutional neural network. By training the target detection model, the target category, bounding box width, bounding box height, bounding box center point position and confidence are obtained. The bounding box represents a rectangle containing the target, and the confidence represents the probability that the target is within the bounding box. At the same time, a confidence threshold is set. For video frames below the confidence threshold, the target category, bounding box width, bounding box height and bounding box center point position output by the video frame are not used, and the target category, bounding box width, bounding box height and bounding box center point position output by the target detection model are replaced by linear interpolation.

[0019] Furthermore, for video frames below the confidence threshold, the target object category, bounding box width, bounding box height and bounding box center point position are obtained by linear interpolation, specifically including: taking the target object category of the high confidence frame closest to the low confidence frame as the target object category of the low confidence frame; taking the nearest high confidence frame before the low confidence frame and the nearest high confidence frame after the low confidence frame as the starting point and end point of the interpolation, and calculating the bounding box width, bounding box height and bounding box center point position of the low confidence frame accordingly.

[0020] Furthermore, sample data of the target detection model is obtained through artificial simulation, and sample labels of the target detection model are obtained through manual annotation.

[0021] Furthermore, before training the target detection model, N video frames of the first target image are pre-trained. During the pre-training process, the N video frames of the first target image are input into a classifier, and the classification space of the classifier represents the category of the target object.

[0022] Furthermore, the loss function of the target detection model includes: a cross entropy loss function for classification, an L1 loss function for the bounding box, and a binary cross entropy loss function for confidence. The total loss function is: L total =λ cls L cls +λ reg L reg +λ conf L conf , where L cls , L reg and L conf They are cross entropy loss function, L1 loss function and binary cross entropy loss function, λ cls , reg and λ conf They are the weight parameters of the cross entropy loss function, L1 loss function and binary cross entropy loss function respectively.

[0023] Furthermore, the tracker is initialized using the result of the first output of the target detection model, the initial position of the target and the width and height of the bounding box are set, and the new position of the target is predicted by the tracking algorithm in each subsequent frame, which is combined with the position of the center point of the bounding box obtained by target detection, the position of the target in the video frame is updated, and the position of the target in each frame is saved in the trajectory list, thereby forming the motion trajectory of the target.

[0024] The present invention also provides a computing device, comprising:

[0025] one or more processors;

[0026] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the system.

[0027] The present invention also provides a computer-readable storage medium, wherein a program is stored in the computer-readable storage medium, and the system is implemented when the program is executed by a processor.

[0028] The beneficial effect of the present invention is that the present invention obtains the position of the center point of the target object's bounding box in each frame of the image by constructing and training a target detection model, and updates the target object position in combination with a tracking algorithm, thereby improving the accuracy of the target object's source position. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a module schematic diagram of a high-altitude falling object tracing system based on artificial intelligence of the present invention;

[0030] Figure 2 It is a flow chart of preprocessing image data according to the present invention.

[0031] In the figure: an image acquisition module 101 , a preprocessing module 102 , a target detection module 103 and a trajectory generation module 104 . DETAILED DESCRIPTION

[0032] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0033] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] like Figure 1-Figure 2 As shown, a high-altitude falling object tracing system based on artificial intelligence includes:

[0035] An image acquisition module 101 is used to acquire image data of a target object;

[0036] The image data includes video data from the appearance of the target object to the disappearance or landing of the target object;

[0037] A preprocessing module 102, which is used to preprocess the image data to obtain a first target image, wherein the first target image includes N consecutive video frames, where N is a custom parameter;

[0038] The target detection module 103 is used to construct a target detection model according to the first target image, wherein the input of the target detection model is the first target image, and the output is a first feature sequence, wherein the nth sequence unit of the first feature sequence represents the target object category, bounding box width, bounding box height, bounding box center point position and confidence of the nth video frame, wherein 1≤n≤N;

[0039] The trajectory generation module 104 is used to generate a motion trajectory of the target object according to the first feature sequence in combination with a tracking algorithm.

[0040] In one embodiment of the present invention, a surveillance camera is deployed at a suitable location outside the building to ensure that the camera can cover the building.

[0041] In one embodiment of the present invention, the pre-processing step includes:

[0042] Step S201, by Perform noise removal on the image data, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and Gx, y represents the Gaussian function value at x, y, that is, the weight of the point;

[0043] Step S202, by The image data is enhanced, where Hu, v represents the frequency response of the filter, Du, v represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter;

[0044] Step S203, by Separate the target object and select it in the video frame, where I t (x, y) represents the video frame image, (x, y) is the spatial coordinate, indicating the pixel position in the image, B(x, y) represents the background model, |I t (x, y)-B(x, y)| represents the pixel difference between each frame image and the background model, T represents the first threshold, preferably, the value range of the first threshold is 30 to 50, M(x, y) represents the target mask, and the target mask consists of 1 and 0, that is, if the pixel difference exceeds the first threshold, the pixel is considered to belong to the target and marked as 1, otherwise it is considered to be part of the background and marked as 0; since the background of the video frame image is not completely static, B is used t+1 (x, y) = α*I t (x,y)+(1-α)*B t (x, y) updates the background model, where B t+1 (x, y) is the updated background model, α represents the first learning rate, which is used to control the speed of background model update, and the value range is between 0 and 1. Preferably, α is set to 0.5;

[0045] Step S204, arranging the image data in reverse chronological order;

[0046] Step S205: normalize the image data using a z-score normalization method.

[0047] In one embodiment of the present invention, the target detection model is constructed based on a convolutional neural network. Preferably, Faster RCNN is used to construct the target detection model. After detecting the target object, Faster RCNN returns the coordinates of the bounding box of the target object. The coordinates of the bounding box are represented by a four-tuple (x1, y1, x2, y2), where x1 and y1 are the coordinates of the upper left corner of the bounding box, x2 and y2 are the coordinates of the lower right corner of the bounding box, the width of the bounding box is represented by x2-x1, the height of the bounding box is represented by y2-y1, and the coordinates of the center point of the bounding box are By training the target detection model, the target category, bounding box width, bounding box height, bounding box center point position and confidence are obtained. The bounding box represents a rectangle containing the target, and the confidence represents the probability that the target is within the bounding box. At the same time, a confidence threshold is set. For video frames below the confidence threshold, the target category, bounding box width, bounding box height and bounding box center point position output by the video frame are not used. The target category, bounding box width, bounding box height and bounding box center point position output by the target detection model are replaced by linear interpolation.

[0048] In one embodiment of the present invention, the target object category, bounding box width, bounding box height and bounding box center point position of the video frame below the confidence threshold are obtained by linear interpolation, specifically including: taking the target object category of the high confidence frame closest to the low confidence frame as the target object category of the low confidence frame; taking the nearest high confidence frame before the low confidence frame and the nearest high confidence frame after the low confidence frame as the starting point and end point of the interpolation, and calculating the bounding box width, bounding box height and bounding box center point position of the low confidence frame accordingly; the calculation formula includes:

[0049]

[0050]

[0051]

[0052]

[0053] Where W i , H i 、x i and i Respectively represent the bounding box width, bounding box height, bounding box center point x-axis coordinate and bounding box center point y-axis coordinate of the low confidence frame, W j1 , H j1 、x j1 and j1 Respectively represent the bounding box width, bounding box height, bounding box center point x-axis coordinate and bounding box center point y-axis coordinate of the starting frame, W j2 , H j2 、x j2 and j2 They represent the bounding box width, bounding box height, bounding box center point x-axis coordinate, and bounding box center point y-axis coordinate of the end frame respectively.

[0054] In one embodiment of the present invention, in special circumstances, the confidence threshold is dynamically adjusted to screen video frames. For example, strong light may cause the confidence of the entire first feature sequence to be significantly low. In this case, the model lowers the confidence threshold to obtain more available data points.

[0055] In one embodiment of the present invention, sample data of the target detection model is obtained by artificial simulation. For example, objects can be thrown in a building under a controlled environment and video data can be recorded using a camera. Sample labels of the target detection model are obtained by manual annotation.

[0056] In one embodiment of the present invention, ImageNet is used to pre-train N video frames of the first target image before training the target detection model. During the pre-training process, the N video frames of the first target image are input into a classifier, and the classification space of the classifier represents the category of the target object.

[0057] In one embodiment of the present invention, the loss function of the target detection model includes: a cross entropy loss function for classification, an L1 loss function for the bounding box, and a binary cross entropy loss function for confidence. The total loss function is: L total =λ cls L cls +λ reg L reg +λ conf L conf , where L cls , L reg and L conf They are cross entropy loss function, L1 loss function and binary cross entropy loss function, λ cls , reg and λ conf They are the weight parameters of the cross entropy loss function, L1 loss function and binary cross entropy loss function respectively.

[0058] In one embodiment of the present invention, the target detection model is used to initialize the tracker for the first time, set the initial position of the target object and the width and height of the bounding box, and in each subsequent frame, Predict the new position of the target object, where represents the predicted position vector of the next frame, and F is the state transfer matrix, which is used to represent how the system changes from the current frame to the next frame. K represents the position vector of the current frame, and combines the position of the center point of the bounding box obtained by target detection to automatically adjust the weight of the new position predicted by tracking and the new position of target detection through the Kalman gain, update the position of the target in the video frame, and save the position of the target in each frame to the trajectory list, thereby forming the motion trajectory of the target. The calculation formula of the Kalman gain is:k+1 =F*P k|k *F T +Q*H T *H*F*P k|k *F T +Q*H T +R -1 , where K k+1 represents the Kalman gain, which is used to balance the prediction value and the detection value. H represents the detection matrix, which maps the position vector to the detection space. Q is the process noise covariance matrix, which represents the noise of the system in the prediction process. R is the detection noise covariance matrix, which represents the noise in the detection process. T is the transposition operation. The calculation formula for updating the position is: in represents the updated position, z k+1 Indicates the target position obtained by target detection in the next frame.

[0059] In one embodiment of the present invention, the location of the source is inferred based on the target object's motion trajectory combined with the structure of the building, and the relevant personnel are criticized, educated or held accountable.

[0060] The embodiment of the present invention further provides a computing device, including: a processor, a memory storing a computer program, and when the computer program is executed by the processor, the system described above is executed. All implementation methods in the above system embodiment are applicable to this embodiment and can achieve the same technical effect.

[0061] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0062] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A high-altitude falling object tracing system based on artificial intelligence, characterized in that: include: An image acquisition module, which is used to acquire image data of a target object; The image data includes video data from the appearance of the target object to the disappearance or landing of the target object; A preprocessing module, which is used to preprocess the image data to obtain a first target image, wherein the first target image includes N consecutive video frames, where N is a custom parameter; A target detection module is used to build a target detection model according to the first target image, wherein the input of the target detection model is the first target image, and the output is a first feature sequence, wherein the nth sequence unit of the first feature sequence represents the target object category, bounding box width, bounding box height, bounding box center point position and confidence of the nth video frame, wherein 1≤n≤N; The trajectory generation module is used to generate the motion trajectory of the target object according to the first feature sequence in combination with the tracking algorithm.

2. According to the artificial intelligence-based high-altitude falling object tracing system of claim 1, it is characterized in that: The preprocessing steps include: Step S201, by Perform noise removal on the image data, where σ is the standard deviation of the Gaussian filter, α represents the weight coefficient of the Gaussian filter, and G(x, y) represents the Gaussian function value at (x, y), that is, the weight of the point; Step S202, by The image data is enhanced, where H(u, v) represents the frequency response of the filter, D(u, v0 represents the distance function in the frequency domain, D0 represents the cutoff frequency, n represents the order of the filter, which is used to control the steepness of the filter near the cutoff frequency, and β represents the weight coefficient of the high-pass filter; Step S203, by Separate the target object and select it in the video frame, where I t (x, y) represents the video frame image, (x, y) is the spatial coordinate, indicating the pixel position in the image, B(x, y) represents the background model, |I t (x, y)-B(x, y)| represents the pixel difference between each frame image and the background model, T represents the first threshold, M(x, y) represents the target mask, and the target mask consists of 1 and 0; Step S204, arranging the image data in reverse chronological order; Step S205: normalize the image data using a z-score normalization method.

3. The high-altitude falling object tracing system based on artificial intelligence according to claim 1 is characterized in that: The target detection model is built based on a convolutional neural network. The target category, bounding box width, bounding box height, bounding box center point position and confidence are obtained by training the target detection model. The bounding box represents a rectangle containing the target, and the confidence represents the probability that the target is within the bounding box. A confidence threshold is set at the same time. For video frames below the confidence threshold, the target category, bounding box width, bounding box height and bounding box center point position output by the video frame are not used. The target category, bounding box width, bounding box height and bounding box center point position output by the target detection model are replaced by linear interpolation.

4. The high-altitude falling object tracing system based on artificial intelligence according to claim 3 is characterized in that: For video frames below the confidence threshold, the target object category, bounding box width, bounding box height and bounding box center point position are obtained by linear interpolation, specifically including: taking the target object category of the high-confidence frame closest to the low-confidence frame as the target object category of the low-confidence frame; taking the nearest high-confidence frame before the low-confidence frame and the nearest high-confidence frame after the low-confidence frame as the starting point and end point of the interpolation, and calculating the bounding box width, bounding box height and bounding box center point position of the low-confidence frame accordingly.

5. The high-altitude falling object tracing system based on artificial intelligence according to claim 1 is characterized in that: The sample data of the target detection model is obtained through artificial simulation, and the sample labels of the target detection model are obtained through manual annotation.

6. The high-altitude falling object tracing system based on artificial intelligence according to claim 1 is characterized in that: Before training the target detection model, N video frames of the first target image are pre-trained. During the pre-training process, the N video frames of the first target image are input into the classifier, and the classification space of the classifier represents the category of the target object.

7. The high-altitude falling object tracing system based on artificial intelligence according to claim 1 is characterized in that: The loss functions of the target detection model include: cross entropy loss function for classification, L1 loss function for bounding box, and binary cross entropy loss function for confidence. The total loss function is: L total =λ cls L cls +λ reg L reg +λ conf L conf , where L cls , L reg and L conf They are cross entropy loss function, L1 loss function and binary cross entropy loss function, λ cls , reg and λ conf They are the weight parameters of the cross entropy loss function, L1 loss function and binary cross entropy loss function respectively.

8. The high-altitude falling object tracing system based on artificial intelligence according to claim 1 is characterized in that: Use the first output of the target detection model to initialize the tracker, set the initial position of the target and the width and height of the bounding box, and use the tracking algorithm to predict the new position of the target in each subsequent frame. Combine it with the position of the center point of the bounding box obtained by target detection, update the position of the target in the video frame, and save the target position of each frame in the trajectory list, thereby forming the motion trajectory of the target.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the system according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed by a processor, the system according to any one of claims 1 to 8 is implemented.

Images