Video processing method, system, apparatus, device and readable storage medium

Abstract

The present disclosure relates to the technical field of video processing, and particularly relates to a video processing method, system, device, equipment and readable storage medium, the method comprising: acquiring a first image frame; generating a first gray value array based on the first image frame; if the first image frame is not a first frame image, acquiring a second gray value array; calculating a relative motion amount; acquiring a first boundary value of an object detected last time, and traversing from a first element in the first gray value array; if a plurality of first target elements located before a current element in the traversal, adjacent to the current element in the traversal, having a value greater than or equal to a preset gray threshold value, and having a quantity greater than or equal to a second quantity threshold value are determined, acquiring a first boundary value and a second boundary value of a new object; cropping a region between a first column of pixels and a second column of pixels in the first image frame, and acquiring a category label of the new object based on the cropped image. The scheme reduces the amount of data to be processed, and improves the security inspection efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video processing, and particularly relates to a video processing method, system, device, equipment and readable storage medium. BACKGROUND

[0002] In recent years, security inspection equipment is widely used in airports, subways and large-scale public activity places. When performing security inspection, the detected object is generally sent into the security inspection equipment through a conveyor belt, and an X-ray beam is emitted by means of a ray source. The X-ray beam penetrates the detected object in the security inspection equipment and falls on an X-ray sensor. Based on the X-ray received by the X-ray sensor, the security inspection equipment can obtain a high-quality video. In general, the object category of the detected object can be determined by relevant staff according to the video, so as to determine whether the detected object is a prohibited object based on the object category. However, due to the heavy work pressure of the staff and the interference, the staff is prone to erroneously determine the object category.

[0003] In recent years, in order to improve the above situation, the object recognition can be performed on each frame of image in the video obtained by the security inspection equipment through a corresponding device, so as to obtain the object category of the object in each frame of image. However, in this scheme, since the recognition needs to be performed on the whole frame of image, the data amount for processing each frame of image is large, thereby reducing the speed of recognizing the object category, resulting in low efficiency of security inspection and damaging the user experience. SUMMARY

[0004] In order to solve the problems in the related art, the embodiments of the present disclosure provide a video processing method, system, device, equipment and readable storage medium.

[0005] In a first aspect, a video processing method is provided in the embodiments of the present disclosure, comprising:

[0006] obtaining a first image frame closest to a current time in a collection time of a video;

[0007] generating a gray-scale image based on the first image frame, and generating a first gray-scale value array based on the gray-scale image, wherein an element value of an element in the first gray-scale value array is a sum of gray-scale values of pixels in a column in the gray-scale image, ;

[0008] if the first image frame is not a first frame of image in the video, obtaining a second gray-scale value array generated based on a second image frame in advance, wherein the second image frame is a previous frame of image of the first image frame;

[0009] traversing calculating , obtaining reference motion quantities​​ wherein is an element value of an element at the i th position in the first gray value array, is an element value of an element at the i th position in the second gray value array, is an element value of an element at the i th position in the second gray value array, is a preset first quantity threshold; determines the minimum value of the i reference motion quantities

[0010] as the relative motion quantity of the pixel between the first image frame and the second image frame .

[0011] obtains a first boundary value of the object that is detected for the last time in the image frame before the first image frame , and traverses from the element at the i th position in the first gray value array forward;

[0012] if a plurality of continuous first target elements are determined in the first gray value array, which are located before and adjacent to the current element in the traversal, have element values greater than or equal to the preset gray threshold, and have a quantity greater than or equal to the second quantity threshold, a first boundary value of a new object detected in the first image frame and a second boundary value are obtained, wherein is a serial number of the first target element in the first gray value array, is a serial number of the current element in the traversal in the first gray value array;

[0013] the region between the j th column pixel and the k th column pixel in the first image frame is cropped, and object detection is performed based on the cropped image to obtain a class label of a new object detected in the first image frame. In an embodiment of the present disclosure, the video processing method further comprises:

[0014] if a plurality of continuous first target elements are not determined in the first gray value array, which are located before and adjacent to the current element in the traversal, have element values greater than or equal to the preset gray threshold, and have a quantity greater than or equal to the second quantity threshold, the first boundary value of the object that is detected for the last time in the image frame before the first image frame is set to .

[0015] In an embodiment of the present disclosure, the video processing method further comprises:

[0016]

[0017] ​​​​​If the first image frame is the first frame image in the video, traversing from the last element in the first gray value array to the front;

[0018] If a plurality of second target elements, which are located before the current element in the traversal, are adjacent to the current element in the traversal, have element values greater than or equal to the preset gray threshold, and have a quantity greater than or equal to the second quantity threshold, are determined in the first gray value array, and the element value of the current element in the traversal is greater than or equal to the preset gray threshold, it is determined that the first boundary value of the object is detected in the first image frame and the second boundary value , wherein is the ordinal of the first second target element in the plurality of second target elements in the first gray value array, is the ordinal of the current element in the traversal in the first gray value array;

[0019] cropping the region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain the class label of the new object detected in the first image frame. cropping the region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain the class label of the new object detected in the first image frame.

[0020] In an embodiment of the present disclosure, generating a gray image based on the first image frame comprises:

[0021] gray processing the first image frame;

[0022] dividing the first image frame after gray processing into a plurality of rectangular regions with consistent sizes, and obtaining the average value of the gray values of the pixels in each rectangular region;

[0023] traversing each pixel in the first image frame after gray processing, if the gray value of the current traversed pixel is greater than or equal to the average value, setting the gray value of the current traversed pixel to 255, if the gray value of the current traversed pixel is less than the average value, setting the gray value of the current traversed pixel to 0, to obtain the gray image.

[0024] In an embodiment of the present disclosure, the video processing method further comprises:

[0025] displaying the first image frame;

[0026] displaying the object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying the class label of the new object detected in the first image frame on the first image frame according to the position of the object boundary.

[0027] In an embodiment of the present disclosure, the video processing method further comprises:

[0028] ​acquiring object tracking data, the object tracking data comprising first boundary values, second boundary values and category labels of objects detected in image frames before the first image frame;

[0029] increasing the first boundary values and the second boundary values of each object in the object tracking data by a relative motion amount, to obtain updated object tracking data;

[0030] displaying, on the first image frame, object boundaries according to the first boundary values and the second boundary values of newly detected objects in the first image frame, and displaying, on the first image frame, category labels of the newly detected objects in the first image frame according to positions of the object boundaries, comprises:

[0031] inserting the first boundary values, the second boundary values and the category labels of the newly detected objects in the first image frame into the updated object tracking data;

[0032] displaying, on the first image frame, boundaries of corresponding objects according to the first boundary values and the second boundary values of the corresponding objects in the inserted object tracking data, and displaying, on the first image frame, category labels of the corresponding objects in the inserted object tracking data according to positions of the boundaries of the corresponding objects.

[0033] In an embodiment of the present disclosure, displaying, on the first image frame, boundaries of corresponding objects according to the first boundary values and the second boundary values of the corresponding objects in the inserted object tracking data, comprises:

[0034] displaying, on the first image frame, a boundary of a rectangular image region comprising pixels in a first column and pixels in a second column in the first image frame, wherein is the first boundary value of the corresponding object in the inserted object tracking data, is the second boundary value of the corresponding object in the inserted object tracking data. In an embodiment of the present disclosure, before inserting the first boundary values, the second boundary values and the category labels of the newly detected objects in the first image frame into the updated object tracking data, the video processing method further comprises:

[0035] If the second boundary value of any object in the updated object tracking data is greater than a maximum pixel column number of the first image frame, removing the first boundary value, the second boundary value and the category label of the any object from the updated object tracking data;

[0036] If the second boundary value of any object in the updated object tracking data is greater than a maximum pixel column number of the first image frame, removing the first boundary value, the second boundary value and the category label of the any object from the updated object tracking data;

[0037] inserting the first boundary values, the second boundary values and the category labels of the newly detected objects in the first image frame into the updated object tracking data, comprises:

[0038] ​The first boundary value, second boundary value, and category label of the new object will be detected in the first image frame, and the object tracking data after removal will be inserted.

[0039] Secondly, this disclosure provides a video processing system, which includes: security inspection equipment, video processing equipment, and object category identification equipment, wherein:

[0040] The security inspection equipment is set to capture video and send it to video processing equipment.

[0041] The video processing device is configured to: acquire the first image frame in the video whose capture time is closest to the current time; generate a grayscale image based on the first image frame; and generate a first grayscale value array based on the grayscale image, wherein the first grayscale value array contains the first image frame whose capture time is closest to the current time. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. If the first image frame is not the first frame in the video, then obtain the second grayscale value array pre-generated based on the second image frame, where the second image frame is the frame preceding the first image frame; iterate through... calculate ,get Reference exercise volume ,in The first grayscale value in the first grayscale array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; Obtain the first boundary value of the most recently detected object in the image frame preceding the first image frame, and extract the first boundary value from the first grayscale array. The process begins by iterating forward from the first element. If, in the first grayscale value array, multiple consecutive first target elements are found that are located before the current element being iterated, adjacent to the current element being iterated, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the new object detected in the first image frame is obtained. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among a series of consecutive first target elements. The index of the current element in the first grayscale array is used for iteration; for the first image frame... Column pixels and the first The region between columns of pixels is cropped, and the cropped image is sent to the object category identification device;

[0042] The object category labeling device is configured to perform object detection based on the cropped image to obtain a first category label for a new object detected in the first image frame.

[0043] In one embodiment of this disclosure, the video processing device is further configured to:

[0044] If no consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, and whose element values ​​are all greater than or equal to a preset grayscale threshold and whose quantity is greater than or equal to a second quantity threshold, then the first boundary value of the most recently detected object in the image frame preceding the first image frame is set to... .

[0045] In one embodiment of this disclosure, the video processing device is further configured to:

[0046] If the first image frame is the first frame of the video, then traverse backwards starting from the last element of the first grayscale value array;

[0047] If, in the first grayscale value array, there are multiple consecutive second target elements that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset grayscale threshold, then the first boundary value of the object detected in the first image frame is determined. With the second boundary value ,in The ordinal number of the first second target element in a series of consecutive second target elements within the first grayscale array. The ordinal number of the current element in the first grayscale array;

[0048] For the first image frame Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

[0049] In one embodiment of this disclosure, generating a grayscale image based on a first image frame includes:

[0050] Perform grayscale processing on the first image frame;

[0051] The first image frame after grayscale processing is divided into multiple rectangular regions of the same size, and the average grayscale value of the pixels in each rectangular region is obtained.

[0052] Traverse each pixel in the first image frame after the grayscale processing, if the grayscale value of the current traversed pixel is greater than or equal to the average value, set the grayscale value of the current traversed pixel to 255, if the grayscale value of the current traversed pixel is less than the average value, set the grayscale value of the current traversed pixel to 0, to obtain a grayscale image.

[0053] In an embodiment of the present disclosure, the video processing system further comprises a display device;

[0054] The video processing device is further configured to:

[0055] send the first boundary value, the second boundary value and the category label of the new object detected in the first image frame to the display device;

[0056] The display device is configured to:

[0057] display the first image frame;

[0058] display the object boundary according to the first boundary value and the second boundary value of the new object detected in the first image frame on the first image frame, and display the category label of the new object detected in the first image frame according to the position of the object boundary on the first image frame.

[0059] In an embodiment of the present disclosure, the video processing device is further configured to:

[0060] obtain object tracking data, the object tracking data comprising the first boundary value, the second boundary value and the category label of the object detected in the image frame before the first image frame;

[0061] increase the first boundary value and the second boundary value of each object in the object tracking data by a relative motion amount, to obtain updated object tracking data;

[0062] send the first boundary value, the second boundary value and the category label of the new object detected in the first image frame to the display device, comprising:

[0063] insert the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, and send the inserted object tracking data to the display device;

[0064] display the object boundary according to the first boundary value and the second boundary value of the new object detected in the first image frame on the first image frame, and display the category label of the new object detected in the first image frame according to the position of the object boundary on the first image frame, comprising:

[0065] displaying a boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and displaying a category label of the corresponding object on the first image frame according to the position of the boundary of the corresponding object.

[0066] In an embodiment of the present disclosure, displaying a boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data comprises:

[0067] displaying a boundary of a rectangular image region between the first column of pixels and the second column of pixels in the first image frame, wherein the first boundary value is a first boundary value of the corresponding object in the inserted object tracking data, the second boundary value is a second boundary value of the corresponding object in the inserted object tracking data.

[0068] In an embodiment of the present disclosure, before inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, the video processing device is further configured to:

[0069] if the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns of the first image frame, removing the first boundary value, the second boundary value and the category label of the object from the updated object tracking data;

[0070] inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data comprises:

[0071] inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the removed object tracking data.

[0072] In a third aspect, an embodiment of the present disclosure provides a video processing device, comprising:

[0073] an image frame acquisition module configured to acquire a first image frame closest to a current time in a video acquisition time;

[0074] a first array acquisition module configured to generate a grayscale image based on the first image frame, and generate a first grayscale value array based on the grayscale image, wherein an element value of an element in the first grayscale value array is a sum of grayscale values of pixels in a column in the grayscale image,

[0075] ​​​​​​The second array obtaining module is configured to obtain a second gray value array generated in advance based on a second image frame if the first image frame is not a first frame image in the video, the second image frame being a previous frame image of the first image frame.

[0076] The relative motion recognition module is configured to traverse the first gray value array. The calculation obtains reference motion amounts , wherein is an element value of an element at a position of the first gray value array, is an element value of an element at a position of the second gray value array, is a preset first quantity threshold; and a minimum value of the reference motion amounts is determined as a relative motion amount of the pixel between the first image frame and the second image frame.

[0077] The first boundary obtaining module is configured to obtain a first boundary value of a last detected object generated in advance , and traverse from an element at a position of the first gray value array.

[0078] The second boundary obtaining module is configured to obtain a first boundary value of a new object detected in the first image frame and a second boundary value if a plurality of continuous first target elements are determined in the first gray value array, the plurality of continuous first target elements being located before a current element in the traversal, adjacent to the current element in the traversal, having element values greater than or equal to a preset gray threshold, and having quantities greater than or equal to a second quantity threshold. is a serial number of a first target element in the plurality of continuous first target elements in the first gray value array, is a serial number of the current element in the traversal in the first gray value array.

[0079] The object detection module is configured to crop a region between a pixel at a column and a pixel at a column in the first image frame, and perform object detection based on the cropped image to obtain a class label of a new object detected in the first image frame.

[0080] In a fourth aspect, an electronic device is provided, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the video processing method in the first aspect of the present disclosure. ​​​​​​

[0081] In a fifth aspect, a computer readable storage medium is provided in the embodiments of the present disclosure, and the computer readable storage medium stores computer instructions. The computer instructions are executed by a processor to implement the video processing method in the first aspect of the present disclosure.

[0082] In a sixth aspect, a computer program product is provided in the embodiments of the present disclosure, and the computer program product includes a computer program. The computer program is executed by a processor to implement the video processing method in the first aspect of the present disclosure.

[0083] According to the technical scheme provided in the embodiments of the present disclosure, since the gray value array indicates the gray characteristics of the pixels in the corresponding column in the corresponding image frame, the data amount is small. Therefore, in the steps of obtaining the relative motion amount of the pixels between the first image frame and the second image frame based on the first gray value array and the second gray value array, and in the step of traversing from the first element in the first gray value array to the front to detect the object that is not detected in the image frame before the first image frame, the data amount required for processing is small. Meanwhile, the above steps can also obtain the relative motion amount with high accuracy and accurately determine the region of the pixels corresponding to the plurality of first target elements in the first image frame, i.e., the region between the first column of pixels and the second column of pixels. By detecting the object in the region in the first image frame to determine the corresponding object category, the data amount required for processing during detection is further reduced. In summary, the above scheme can accurately identify the object type of the object that is not detected before in the first image frame, reduce the data amount required for processing, improve the speed of identifying the object category, help to improve the efficiency of security check, and thus improve the user experience.

[0084] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0085] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings. In the drawings:

[0086] Figure 1 A flowchart of a video processing method according to an embodiment of the present disclosure is shown;

[0087] Figure 2 A structural block diagram of a video processing system according to an embodiment of the present disclosure is shown;

[0088] Figure 3 A structural block diagram of a video processing apparatus according to an embodiment of the present disclosure is shown;​​​​​

[0089] Figure 4 A structural block diagram of an electronic device according to an embodiment of the disclosure is shown;

[0090] Figure 5 A structural schematic diagram of a computer system suitable for implementing a method according to an embodiment of the disclosure is shown. DETAILED DESCRIPTION

[0091] Hereinafter, exemplary embodiments of the disclosure will be described in detail with reference to the accompanying drawings so as to be easily implemented by those skilled in the art. Also, parts irrelevant to the description of the exemplary embodiments are omitted in the accompanying drawings for the sake of clarity.

[0092] In the disclosure, it should be understood that terms such as "include" or "have" are intended to indicate that there are features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the possibility of adding one or more other features, numbers, steps, actions, components, parts or combinations thereof.

[0093] It is also necessary to note that the embodiments in the disclosure and the features in the embodiments can be combined with each other without conflict. The disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0094] In the disclosure, if it involves an operation of acquiring user information or user data or an operation of showing user information or user data to others, the operation is an operation authorized, confirmed by the user, or actively selected by the user.

[0095] In recent years, in order to avoid relevant personnel carrying prohibited objects into public places such as airports, subways and large-scale public activities places which require high security level, security inspection equipment can be used to detect objects carried by relevant personnel.

[0096] In related technologies, during inspection, the object to be inspected is placed at the entrance of the conveyor belt of the security inspection equipment. The conveyor belt automatically sends the object into the X-ray scanning area inside the machine. An X-ray source inside the equipment emits an X-ray beam that penetrates the object and falls onto an X-ray sensor located behind it. The X-ray sensor converts the received X-rays into electrical signals, and the security inspection equipment obtains an image based on these signals. Multiple frames of images can then be used to generate a video. However, because objects have different densities, the attenuation of the X-ray beam penetrating the object varies. Therefore, in the resulting video, different density regions of the object will appear differently (e.g., darker colors represent higher density, and lighter colors represent lower density). By reviewing this video, staff can determine the object's category and whether it is a prohibited item. However, this process is often stressful and prone to interference for staff, which can easily lead to incorrect object classification.

[0097] In recent years, to improve the above situation, corresponding devices can be used to perform object recognition on each frame of the video obtained by security inspection equipment, thereby obtaining the object category of the object in each frame. However, in this solution, in order to avoid misidentification or missed identification, recognition needs to be performed on the entire frame, resulting in a large amount of data to be processed, which reduces the speed of object category recognition, leading to low security inspection efficiency and impairing user experience.

[0098] To address the aforementioned problems, this disclosure provides a video processing method, system, apparatus, device, and readable storage medium.

[0099] In the technical solution provided in this disclosure embodiment, a first image frame from the video whose acquisition time is closest to the current time is obtained; a grayscale image is generated based on the first image frame, and a first grayscale value array is generated based on the grayscale image, wherein the first grayscale value array contains the first image frame whose acquisition time is closest to the current time. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. If the first image frame is not the first frame in the video, then obtain the second grayscale value array pre-generated based on the second image frame, where the second image frame is the frame preceding the first image frame; iterate through... calculate ,get Reference exercise volume ,in The first grayscale value in the first grayscale array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; Obtain the first boundary value of the most recently detected object in the image frames preceding the first image frame. and from the first grayscale value array the first The first element is traversed forward; if multiple consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the new object detected in the first image frame is obtained. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among a series of consecutive first target elements. The index of the current element in the first grayscale array is used for iteration; for the first image frame... Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

[0100] According to the technical solution provided in this disclosure, since the grayscale value array has a small amount of data while indicating the grayscale features of pixels in the corresponding column of the corresponding image frame, the relative motion of pixels between the first image frame and the second image frame is obtained based on the first grayscale value array and the second grayscale value array. In the steps, and in the first grayscale value array The steps of traversing each element forward to detect objects not detected in image frames prior to the first image frame require relatively little data processing, and these steps can also obtain relatively accurate relative motion data. And to more accurately determine the region where the pixels corresponding to multiple first target elements are located in the first image frame, i.e., the first Column pixels and the first The area between columns of pixels is targeted for object detection only in the first image frame to determine the corresponding object category, further reducing the amount of data that needs to be processed during detection. In summary, the above scheme reduces the amount of data required for processing while accurately identifying the object type of objects that were not previously detected in the first image frame, thereby improving the speed of object category identification, enhancing security check efficiency, and ultimately improving the user experience.

[0101] Figure 1 A flowchart illustrating a video processing method according to an embodiment of the present disclosure is shown.Figure 1 As shown, the video processing method includes the following steps S101-S107:

[0102] In step S101, the first image frame in the video whose acquisition time is closest to the current time is obtained;

[0103] In one implementation of this disclosure, the acquired video can be in Low-Voltage Differential Signaling (LVDS) format. The first image frame can be determined based on the current time and the acquisition time of each image frame in the video.

[0104] In step S102, a grayscale image is generated based on the first image frame, and a first grayscale value array is generated based on the grayscale image;

[0105] The first grayscale value array contains the first... The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels;

[0106] in, ;

[0107] In step S103, if the first image frame is not the first frame image in the video, then the second grayscale value array generated in advance based on the second image frame is obtained;

[0108] The second image frame is the image frame preceding the first image frame;

[0109] In one implementation of this disclosure, in the second grayscale value array, the first The element value of the nth element is the value of the grayscale image generated based on the second image frame. The sum of the grayscale values ​​of the column pixels.

[0110] In step S104, traverse calculate ,get Reference exercise volume and will Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ;

[0111] in The first grayscale value in the first grayscale array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold;

[0112] In one implementation of this disclosure, the first quantity threshold This can be understood as the maximum distance a pixel moves between two adjacent frames. For example, .

[0113] In step S105, the first boundary value of the most recently detected object in the image frame preceding the first image frame is obtained. and from the first grayscale value array the first Iterate forward through each element;

[0114] In one implementation of this disclosure, the detected object moves from left to right in the video as an example. The first boundary value is used to indicate the column number of the leftmost pixel in the object boundary of the most recently detected object in the second image frame.

[0115] In step S106, if a plurality of consecutive first target elements are determined in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the new object detected in the first image frame is obtained. With the second boundary value ;

[0116] in The ordinal number of the first target element in the first grayscale value array among a series of consecutive first target elements. The ordinal number of the current element in the first grayscale array;

[0117] In one implementation of this disclosure, the preset grayscale threshold is 64.

[0118] In step S107, the first image frame is... Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

[0119] According to the technical solution provided in this disclosure, since the grayscale value array has a small amount of data while indicating the grayscale features of pixels in the corresponding column of the corresponding image frame, the relative motion of pixels between the first image frame and the second image frame is obtained based on the first grayscale value array and the second grayscale value array. In the steps, and in the first grayscale value array The steps of traversing each element forward to detect objects not detected in image frames prior to the first image frame require relatively little data processing, and these steps can also obtain relatively accurate relative motion data. And to more accurately determine the region where the pixels corresponding to multiple first target elements are located in the first image frame, i.e., the first Column pixels and the first The area between columns of pixels is targeted for object detection only in the first image frame to determine the corresponding object category, further reducing the amount of data that needs to be processed during detection. In summary, the above scheme reduces the amount of data required for processing while accurately identifying the object type of objects that were not previously detected in the first image frame, thereby improving the speed of object category identification, enhancing security check efficiency, and ultimately improving the user experience.

[0120] In one embodiment of this disclosure, the video processing method further includes the following steps:

[0121] If no consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, and whose element values ​​are all greater than or equal to a preset grayscale threshold and whose quantity is greater than or equal to a second quantity threshold, then the first boundary value of the most recently detected object in the image frame preceding the first image frame is set to... .

[0122] In the technical solution provided in this disclosure, if no consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, and whose element values ​​are all greater than or equal to a preset grayscale threshold and whose quantity is greater than or equal to a second quantity threshold, then it is determined that no new object has been detected in the first image frame. This is achieved by setting the first boundary value of the most recently detected object in an image frame preceding the first image frame to... In subsequent steps, based on the aforementioned first boundary value, it can determine that the leftmost pixel on the object boundary of the most recently detected object is located in the first image frame. The columns of pixels facilitate subsequent detection processes.

[0123] In one embodiment of this disclosure, the video processing method further includes:

[0124] If the first image frame is the first frame of the video, then traverse backwards starting from the last element of the first grayscale value array;

[0125] If, in the first grayscale value array, there are multiple consecutive second target elements that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset grayscale threshold, then the first boundary value of the object detected in the first image frame is determined. With the second boundary value ,in a serial number of the first target element in the first gray value array, a serial number of the current element in the first gray value array;

[0126] cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a category label of a new object detected in the first image frame. cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a category label of a new object detected in the first image frame.

[0127] In the technical scheme provided in the embodiments of the present disclosure, if the first image frame is a first frame image in a video, it is determined that no object is detected in the video at the current moment, and in order to avoid missing detection, the last element in the first gray value array is traversed forward to detect the object in the first image frame. Wherein, when the current element in the first gray value array is determined to be located before the current element in the traversal, adjacent to the current element in the traversal, the element value is greater than or equal to the preset gray threshold, and the number is greater than or equal to the second number threshold, and the element value of the current element in the traversal is greater than or equal to the preset gray threshold, it is determined that the region corresponding to the continuous multiple second target elements in the first image frame, that is, the region between the first column of pixels and the second column of pixels in the first image frame, detects the object, so by cropping the region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image, a category label of a new object detected in the first image frame is obtained. Wherein, since the first gray value array can indicate the gray feature of the pixels corresponding to the column in the first image frame, and the data amount is small, the region corresponding to the object in the first image frame can be accurately determined based on the first gray value array, so as to detect the object category for the region, therefore, the above scheme reduces the amount of data required to be processed under the premise of accurately identifying the object type of the object in the first image frame, thereby improving the speed of identifying the object category, which helps to improve the efficiency of security check and improves the user experience. cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a category label of a new object detected in the first image frame. cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a category label of a new object detected in the first image frame.

[0128] In one embodiment of the present disclosure, generating a gray image based on the first image frame comprises:

[0129] performing gray processing on the first image frame;

[0130] dividing the first image frame after gray processing into a plurality of rectangular regions with consistent sizes, and obtaining an average value of the gray values of the pixels in each rectangular region;

[0131] ​​​Traverse each pixel in the first image frame after the gray processing, if the gray value of the current traversed pixel is greater than or equal to the average value, set the gray value of the current traversed pixel to 255, if the gray value of the current traversed pixel is less than the average value, set the gray value of the current traversed pixel to 0, to obtain a gray image.

[0132] In the technical scheme provided by the embodiment of the present disclosure, by dividing the first image frame after the gray processing into a plurality of rectangular regions with consistent sizes, obtaining the average value of the gray values of the pixels in each rectangular region, and traversing each pixel in the first image frame after the gray processing, if the gray value of the current traversed pixel is greater than or equal to the average value, setting the gray value of the current traversed pixel to 255, if the gray value of the current traversed pixel is less than the average value, setting the gray value of the current traversed pixel to 0, the clutter in the first image frame after the gray processing can be filtered out, while the image features in the first image frame are preserved as much as possible, thereby reducing the difficulty of detecting the category of the object in the corresponding image region, and helping to further improve the efficiency of security inspection.

[0133] In an embodiment of the present disclosure, the video processing method further comprises:

[0134] displaying the first image frame;

[0135] displaying the object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying the category label of the new object detected in the first image frame according to the position of the object boundary on the first image frame.

[0136] In the technical scheme provided by the embodiment of the present disclosure, by displaying the first image frame, displaying the object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying the category label of the new object detected in the first image frame according to the position of the object boundary on the first image frame, the position of the object boundary of the new object detected in the first image frame and the category label can be more intuitively obtained by the staff, and the user experience is improved.

[0137] In an embodiment of the present disclosure, the video processing method further comprises:

[0138] obtaining object tracking data, the object tracking data comprising the first boundary value, the second boundary value and the category label of the object detected in the image frame before the first image frame;

[0139] increasing the first boundary value and the second boundary value of each object in the object tracking data by a relative motion amount to obtain updated object tracking data;

[0140] displaying, on the first image frame, a boundary of the object according to the first boundary value and the second boundary value of the object detected in the first image frame, and displaying, on the first image frame, a category label of the object detected in the first image frame according to a position of the boundary of the object, comprising:

[0141] inserting the first boundary value, the second boundary value, and the category label of the object detected in the first image frame into the updated object tracking data;

[0142] displaying, on the first image frame, a boundary of the corresponding object according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and displaying, on the first image frame, a category label of the corresponding object in the inserted object tracking data according to a position of the boundary of the corresponding object.

[0143] In an implementation manner of the present disclosure, the displaying, on the first image frame, the boundary of the corresponding object according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data can be implemented by the following manner: displaying, on the first image frame, a boundary of a rectangular image region between a first column of pixels and a second column of pixels in the first image frame, wherein the first boundary value of the corresponding object in the inserted object tracking data is a first boundary value of the rectangular image region, and the second boundary value of the corresponding object in the inserted object tracking data is a second boundary value of the rectangular image region.

[0144] In the technical solution provided by the embodiments of the present disclosure, by obtaining the object tracking data, increasing the first boundary value and the second boundary value of each object in the object tracking data by the relative motion amount, and obtaining the updated object tracking data, it can be ensured that the first boundary value and the second boundary value of the object in the updated object tracking data can indicate the position of the object boundary of the object in the first image frame. By inserting the first boundary value, the second boundary value, and the category label of the object detected in the first image frame into the updated object tracking data, displaying the boundary of the corresponding object according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data on the first image frame, and displaying the category label of the corresponding object in the inserted object tracking data according to the position of the boundary of the corresponding object on the first image frame, it can be convenient for the staff to more intuitively know the position of the object boundary of the currently detected object (including the object detected in the first image frame and the object detected in the image frame before the first image frame) in the first image frame and the corresponding category label, thereby improving the user experience.

[0145] In an implementation manner of the present disclosure, before the inserting the first boundary value, the second boundary value, and the category label of the object detected in the first image frame into the updated object tracking data, the video processing method further comprises:​​​​

[0146] If the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns in the first image frame, then the first boundary value, the second boundary value, and the category label of any object will be removed from the updated object tracking data.

[0147] The first boundary value, second boundary value, and category label of the new object detected in the first image frame will be inserted into the updated object tracking data, including:

[0148] The first boundary value, second boundary value, and category label of the new object will be detected in the first image frame, and the object tracking data after removal will be inserted.

[0149] In the technical solution provided in this disclosure, when the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns in the first image frame, the boundary of the object can be determined to exceed the image range of the first image frame. In this situation, by removing the first boundary value, second boundary value, and category label of any object from the updated object tracking data, it can be ensured that the removed object tracking data no longer includes data related to the object, and the object boundary and category label of the object will not be displayed on the first image frame in subsequent steps, thereby avoiding interference with staff and improving the user experience.

[0150] Figure 2 A structural block diagram of a video processing system according to an embodiment of the present disclosure is shown. Figure 2 As shown, the video processing system includes: security inspection equipment 201, video processing equipment 202, and object category identification equipment 203, wherein:

[0151] Security inspection equipment 201 is configured to: capture video and send the video to video processing equipment 202;

[0152] The video processing device 202 is configured to: acquire the first image frame in the video whose acquisition time is closest to the current time; generate a grayscale image based on the first image frame; and generate a first grayscale value array based on the grayscale image, wherein the first grayscale value array contains the first image frame whose acquisition time is closest to the current time. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. If the first image frame is not the first frame in the video, then obtain the second grayscale value array pre-generated based on the second image frame, where the second image frame is the frame preceding the first image frame; iterate through... calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; the... Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; Obtain the first boundary value of the most recently detected object in the image frame preceding the first image frame, and extract the first boundary value from the first grayscale array. The process begins by iterating forward from the first element. If, in the first grayscale value array, multiple consecutive first target elements are found that are located before the current element being iterated, adjacent to the current element being iterated, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the new object detected in the first image frame is obtained. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among a series of consecutive first target elements. The index of the current element in the first grayscale array is used for iteration; for the first image frame... Column pixels and the first The region between the columns of pixels is cropped, and the cropped image is sent to the object category identification device 203;

[0153] The object category labeling device 203 is configured to perform object detection based on the cropped image to obtain a first category label for a new object detected in the first image frame.

[0154] According to the technical solution provided in this disclosure, since the grayscale value array has a small amount of data while indicating the grayscale features of pixels in the corresponding column of the corresponding image frame, the relative motion of pixels between the first image frame and the second image frame is obtained based on the first grayscale value array and the second grayscale value array. In the steps, and in the first grayscale value array The steps of traversing each element forward to detect objects not detected in image frames prior to the first image frame require relatively little data processing, and these steps can also obtain relatively accurate relative motion data. And to more accurately determine the region where the pixels corresponding to multiple first target elements are located in the first image frame, i.e., the first Column pixels and the first The area between the column pixels further reduces the amount of data to be processed during detection by performing object detection on the area in the first image frame only to determine the corresponding object category. In summary, the above scheme reduces the amount of data to be processed while accurately identifying the object type of the object not previously detected in the first image frame, thereby improving the speed of identifying the object category, helping to improve the efficiency of security checks, and thus improving the user experience.

[0155] In an embodiment of the present disclosure, the video processing device 202 is further configured to:

[0156] If a plurality of continuous first target elements, which are located before and adjacent to the current element being traversed in the first gray value array, have element values greater than or equal to the preset gray threshold and a quantity greater than or equal to the second quantity threshold, the first boundary value of the object detected in the image frame before the first image frame is set to the first column pixel of the first image frame. .

[0157] In the technical scheme provided in the embodiments of the present disclosure, if a plurality of continuous first target elements, which are located before and adjacent to the current element being traversed in the first gray value array, have element values greater than or equal to the preset gray threshold and a quantity greater than or equal to the second quantity threshold, it is determined that no new object is detected in the first image frame. In order to ensure that when object detection is performed on the image frame after the first image frame, the first boundary value of the object detected in the image frame before the first image frame is set to the first column pixel of the first image frame, the leftmost pixel in the object boundary of the object detected last time can be determined to be located in the first column pixel of the first image frame in the subsequent step based on the first boundary value. , facilitating the execution of the subsequent detection process.

[0158] In an embodiment of the present disclosure, the video processing device 202 is further configured to:

[0159] If the first image frame is the first image in the video, the last element in the first gray value array is traversed forward;

[0160] If a plurality of continuous second target elements, which are located before and adjacent to the current element being traversed in the first gray value array, have element values greater than or equal to the preset gray threshold and a quantity greater than or equal to the second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset gray threshold, it is determined that the first boundary value of the object detected in the first image frame is and the second boundary value , wherein is the ordinal number of the first second target element in the first gray value array.​ The ordinal number of the current element in the first grayscale array;

[0161] For the first image frame Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

[0162] In the technical solution provided in this disclosure embodiment, if the first image frame is the first frame image in a video, it is determined that no object is detected in the video at the current moment. To avoid missed detections, the system traverses backward from the last element of the first grayscale value array to detect objects in the first image frame. Specifically, if a series of consecutive second target elements are determined in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset grayscale threshold, then the region corresponding to the series of second target elements in the first image frame is determined to be the first image frame containing the first target element. Column pixels and the first An object was detected in the region between columns of pixels, therefore, by analyzing the first image frame... Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of new objects detected in the first image frame. Since the first grayscale value array can indicate the grayscale features of pixels in the corresponding column in the first image frame, and the data volume is relatively small, the region corresponding to the object in the first image frame can be determined relatively accurately based on the first grayscale value array. Therefore, object category detection is performed on that region. Thus, the above scheme reduces the amount of data to be processed while accurately identifying the object type in the first image frame, thereby improving the speed of object category identification, contributing to improved security check efficiency, and enhancing the user experience.

[0163] In one embodiment of this disclosure, generating a grayscale image based on a first image frame includes:

[0164] Perform grayscale processing on the first image frame;

[0165] The first image frame after grayscale processing is divided into multiple rectangular regions of the same size, and the average grayscale value of the pixels in each rectangular region is obtained.

[0166] The gray scale image is obtained by traversing each pixel in the first image frame after the gray scale processing, setting the gray scale value of the current traversed pixel to 255 if the gray scale value of the current traversed pixel is greater than or equal to the average value, and setting the gray scale value of the current traversed pixel to 0 if the gray scale value of the current traversed pixel is less than the average value.

[0167] In the technical scheme provided in the embodiments of the present disclosure, by dividing the first image frame after the gray scale processing into a plurality of rectangular regions with consistent sizes, obtaining the average value of the gray scale values of the pixels in each rectangular region, and traversing each pixel in the first image frame after the gray scale processing, setting the gray scale value of the current traversed pixel to 255 if the gray scale value of the current traversed pixel is greater than or equal to the average value, and setting the gray scale value of the current traversed pixel to 0 if the gray scale value of the current traversed pixel is less than the average value, the clutter in the first image frame after the gray scale processing can be filtered out, while the image features in the first image frame are preserved as much as possible, thereby reducing the difficulty of detecting the category of the object in the corresponding image region, and helping to further improve the efficiency of security inspection.

[0168] In one embodiment of the present disclosure, as shown in Figure 2 The video processing system further includes a display device 204;

[0169] The video processing device 202 is further configured to:

[0170] The first boundary value, the second boundary value, and the category label of the new object detected in the first image frame are sent to the display device 204;

[0171] The display device 204 is configured to:

[0172] The first image frame is displayed.

[0173] The object boundary is displayed on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and the category label of the new object detected in the first image frame is displayed on the first image frame according to the position of the object boundary.

[0174] In the technical scheme provided in the embodiments of the present disclosure, by displaying the first image frame, displaying the object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying the category label of the new object detected in the first image frame on the first image frame according to the position of the object boundary, the position of the object boundary of the new object detected in the first image frame and the category label can be intuitively obtained by the staff, and the user experience is improved.

[0175] In one embodiment of the present disclosure, the video processing device 202 is further configured to:

[0176] Obtaining object tracking data, the object tracking data comprising a first boundary value, a second boundary value and a category label of an object detected in an image frame before the first image frame;

[0177] Increasing the first boundary value and the second boundary value of each object in the object tracking data by a relative motion amount to obtain updated object tracking data;

[0178] Sending the first boundary value, the second boundary value and the category label of the new object detected in the first image frame to the display device 204, comprising:

[0179] Inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, and sending the inserted object tracking data to the display device 204;

[0180] Displaying the object boundary according to the first boundary value and the second boundary value of the new object detected in the first image frame on the first image frame, and displaying the category label of the new object detected in the first image frame according to the position of the object boundary on the first image frame, comprising:

[0181] Displaying the boundary of the corresponding object according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data on the first image frame, and displaying the category label of the corresponding object in the inserted object tracking data according to the position of the boundary of the corresponding object on the first image frame.

[0182] In an implementation manner of the present disclosure, displaying the boundary of the corresponding object according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data on the first image frame can be implemented by the following manner:

[0183] Displaying the boundary of the rectangular image region comprising the first column of pixels and the second column of pixels in the first image frame, wherein the first boundary value of the corresponding object in the inserted object tracking data is the first column of pixels, and the second boundary value of the corresponding object in the inserted object tracking data is the second column of pixels. Displaying the boundary of the rectangular image region comprising the first column of pixels and the second column of pixels in the first image frame, wherein the first boundary value of the corresponding object in the inserted object tracking data is the first column of pixels, and the second boundary value of the corresponding object in the inserted object tracking data is the second column of pixels. Displaying the boundary of the rectangular image region comprising the first column of pixels and the second column of pixels in the first image frame, wherein the first boundary value of the corresponding object in the inserted object tracking data is the first column of pixels, and the second boundary value of the corresponding object in the inserted object tracking data is the second column of pixels.

[0184] ​​In the technical scheme provided in the embodiments of the present disclosure, by obtaining object tracking data, increasing the first boundary value and the second boundary value of each object in the object tracking data by a relative motion amount, and obtaining updated object tracking data, it can be ensured that the first boundary value and the second boundary value of the object in the updated object tracking data can indicate the position of the object boundary of the object in the first image frame; by inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, and displaying the boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and displaying the category label of the corresponding object in the inserted object tracking data on the first image frame according to the position of the boundary of the corresponding object, it can be convenient for the staff to more intuitively know the position of the object boundary of the currently detected object (including the new object detected in the first image frame and the object detected in the image frame before the first image frame) in the first image frame and the corresponding category label, thereby improving the user experience.

[0185] In an embodiment of the present disclosure, before inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, the video processing device 202 is further configured to:

[0186] If the second boundary value of any object in the updated object tracking data is greater than the maximum pixel column number of the first image frame, the first boundary value, the second boundary value and the category label of the object are removed from the updated object tracking data;

[0187] Inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data comprises:

[0188] Inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the removed object tracking data.

[0189] In the technical scheme provided in the embodiments of the present disclosure, when the second boundary value of any object in the updated object tracking data is greater than the maximum pixel column number of the first image frame, it can be determined that the boundary of the object exceeds the image range of the first image frame. In this case, by removing the first boundary value, the second boundary value and the category label of the object from the updated object tracking data, it can be ensured that the removed object tracking data no longer includes data related to the object, and the object boundary and the category label of the object will not be displayed on the first image frame in the subsequent steps, thereby avoiding interference to the staff and improving the user experience.

[0190] Figure 3A structural block diagram of a video processing apparatus according to an embodiment of the present disclosure is shown. This apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0191] like Figure 3 As shown, the video processing device 300 includes:

[0192] The image frame acquisition module 301 is configured to acquire the first image frame in the video whose acquisition time is closest to the current time.

[0193] The first array acquisition module 302 is configured to generate a grayscale image based on a first image frame, and generate a first grayscale value array based on the grayscale image. The first grayscale value array contains the first grayscale value array... The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. ;

[0194] The second array acquisition module 303 is configured to acquire a second grayscale value array pre-generated based on the second image frame if the first image frame is not the first image frame in the video, wherein the second image frame is the previous image frame of the first image frame.

[0195] The relative motion recognition module 304 is set to traverse. calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; the... Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. The first boundary acquisition module 305 is configured to acquire the first boundary value of the most recently detected object, which is generated in advance. and from the first grayscale value array the first Iterate forward through each element;

[0196] The second boundary acquisition module 306 is configured to acquire the first boundary value of the detected new object in the first image frame if, in the first grayscale value array, there are a series of first target elements that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among a series of consecutive first target elements. The ordinal number of the current element in the first grayscale array;

[0197] The object detection module 307 is configured to detect the first image frame in the first image frame. Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

[0198] This disclosure also discloses an electronic device. Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0199] like Figure 4 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to embodiments of the present disclosure.

[0200] This disclosure provides a video processing method, including:

[0201] Obtain the first image frame in the video whose capture time is closest to the current time.

[0202] A grayscale image is generated based on the first image frame, and a first grayscale value array is generated based on the grayscale image, wherein the first grayscale value array contains the first grayscale value... The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. ;

[0203] If the first image frame is not the first image frame in the video, then obtain the second grayscale value array pre-generated based on the second image frame, where the second image frame is the image frame preceding the first image frame;

[0204] Traversal calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold;

[0205] The Reference exercise volume the minimum value in the first gray value array is determined as the relative motion amount of the pixel between the first image frame and the second image frame ; a first boundary value of an object detected last time in an image frame before the first image frame is obtained , and the first boundary value is obtained from the first element in the first gray value array and traversed forward;

[0206] If a plurality of continuous first target elements are determined in the first gray value array, which are located before and adjacent to the current element of the traversal, have element values greater than or equal to a preset gray threshold value, and have a quantity greater than or equal to a second quantity threshold value, the first boundary value of a new object detected in the first image frame is obtained and the second boundary value , wherein is the ordinal of the first first target element in the first gray value array, is the ordinal of the current element of the traversal in the first gray value array;

[0207] The region between the pixels in the first column and the pixels in the second column in the first image frame is cropped, and object detection is performed based on the cropped image to obtain a class label of a new object detected in the first image frame. In an embodiment of the present disclosure, the video processing method further comprises:

[0208] If a plurality of continuous first target elements are not determined in the first gray value array, which are located before and adjacent to the current element of the traversal, have element values greater than or equal to a preset gray threshold value, and have a quantity greater than or equal to a second quantity threshold value, the first boundary value of an object detected last time in an image frame before the first image frame is set to .

[0209] In an embodiment of the present disclosure, the video processing method further comprises:

[0210] If the first image frame is the first frame image in the video, the traversal is started from the last element in the first gray value array and proceeds forward;

[0211] If a plurality of continuous second target elements are determined in the first gray value array, which are located before and adjacent to the current element of the traversal, have element values greater than or equal to a preset gray threshold value, and have a quantity greater than or equal to a second quantity threshold value, and the element value of the current element of the traversal is greater than or equal to the preset gray threshold value, the first boundary value of an object detected in the first image frame is determined and the second boundary value

[0212] , wherein is the ordinal of the first first target element in the first gray value array, is the ordinal of the current element of the traversal in the first gray value array; ​an ordinal of the first target element in the first gray value array, an ordinal of the current element in the first gray value array;

[0213] cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a class label of a new object detected in the first image frame. cropping a region between the first column of pixels and the second column of pixels in the first image frame, and performing object detection based on the cropped image to obtain a class label of a new object detected in the first image frame.

[0214] In an embodiment of the present disclosure, generating a gray image based on the first image frame comprises:

[0215] performing gray processing on the first image frame;

[0216] dividing the first image frame after the gray processing into a plurality of rectangular regions of uniform size, and obtaining an average value of gray values of pixels in each rectangular region;

[0217] traversing each pixel in the first image frame after the gray processing, if a gray value of a currently traversed pixel is greater than or equal to the average value, setting the gray value of the currently traversed pixel to 255, if the gray value of the currently traversed pixel is less than the average value, setting the gray value of the currently traversed pixel to 0, to obtain the gray image.

[0218] In an embodiment of the present disclosure, the video processing method further comprises:

[0219] displaying the first image frame;

[0220] displaying an object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying a class label of the new object detected in the first image frame according to a position of the object boundary on the first image frame.

[0221] In an embodiment of the present disclosure, the video processing method further comprises:

[0222] obtaining object tracking data, the object tracking data comprising a first boundary value, a second boundary value and a class label of an object detected in an image frame before the first image frame;

[0223] increasing the first boundary value and the second boundary value of each object in the object tracking data by a relative motion amount to obtain updated object tracking data;

[0224] displaying an object boundary on the first image frame according to the first boundary value and the second boundary value of the new object detected in the first image frame, and displaying a class label of the new object detected in the first image frame according to a position of the object boundary on the first image frame, comprises: ​

[0225] inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data;

[0226] displaying the boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and displaying the category label of the corresponding object in the inserted object tracking data on the first image frame according to the position of the boundary of the corresponding object.

[0227] In an embodiment of the present disclosure, displaying the boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data includes:

[0228] displaying the boundary of the corresponding object on the first image frame according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data includes: displaying the boundary of the rectangular image region between the first column of pixels and the second column of pixels in the first image frame, wherein the first boundary value is the first boundary value of the corresponding object in the inserted object tracking data, the second boundary value is the second boundary value of the corresponding object in the inserted object tracking data.

[0229] In an embodiment of the present disclosure, before inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data, the video processing method further includes:

[0230] if the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns of the first image frame, removing the first boundary value, the second boundary value and the category label of the any object from the updated object tracking data;

[0231] inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the updated object tracking data includes:

[0232] inserting the first boundary value, the second boundary value and the category label of the new object detected in the first image frame into the removed object tracking data.

[0233] Figure 5 A structural schematic diagram of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0234] As Figure 5 ​As shown, the computer system includes a processing unit that can execute the various methods described above in the embodiments. The processing unit can be a general -purpose central processing unit (CPU), processor in application specific integrated circuits (ASICs), or any other processor of device. The computer system also includes memory or other article to store data. The memory is a read-and-write memory device. The ROM is a read-only memory device that stores static data and instructions that are needed by the processing unit and other modules of the computer system. The permanent storage device, on the other hand, can be a read-and-write memory device that stores instructions and data that are needed by the processing unit for example. Like the memory, the permanent storage device can also be a read-and-write memory device; however, it is a durable good, such as a ROM, EPROM, EEPROM, or floppy disk for example. The working memory, the ROM, and the permanent storage device are examples of articles of manufacture.

[0235] Connected to the bus via an input / output (I / O) interface are an input portion, including keyboard, mouse, etc.; an output portion, including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker; a storage portion, including a hard disk drive, etc.; and a communication portion, including a network interface card, a modem, etc. The communication portion performs communication processes via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive as needed, and a computer program read out from the removable media is installed in the storage portion as needed. The processing unit can be implemented as a CPU, a GPU, a TPU, a FPGA, a NPU, etc.

[0236] In particular, the methods described above can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine readable medium, the computer program containing program code for executing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion, and / or installed from a removable media.

[0237] The flow and block diagrams in the drawings show possible architectures, functional and operational, for systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0238] The units or modules described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of programmable hardware. The described units or modules can also be arranged in a processor, and the name of the unit or module does not constitute a limitation on the unit or module itself in some cases.

[0239] As another aspect, the present disclosure also provides a computer readable storage medium, which can be the computer readable storage medium contained in the electronic device or computer system in the above embodiments; or can exist independently and not be assembled into the device. The computer readable storage medium stores one or more programs, which are used by one or more processors to execute the method described in the present disclosure.

[0240] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

Claims

1. A video processing method, characterized in that, include: Obtain the first image frame in the video whose capture time is closest to the current time. A grayscale image is generated based on the first image frame, and a first grayscale value array is generated based on the grayscale image, wherein the first grayscale value array contains a number of grayscale values. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. ; If the first image frame is not the first frame image in the video, then obtain the second grayscale value array pre-generated based on the second image frame, wherein the second image frame is the previous frame image of the first image frame; Traversal calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; The Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; Obtain the first boundary value of the most recently detected object in the image frames preceding the first image frame. and from the first grayscale value array the first The elements are traversed forward, where the detected object moves in the video from left to right, and the first boundary value... Used to indicate the column number of the leftmost pixel in the object boundary of the most recently detected object in the second image frame; If, in the first grayscale value array, a plurality of consecutive first target elements are determined that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the new object detected in the first image frame is obtained. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among the consecutive plurality of first target elements. This is the ordinal number of the current element in the first grayscale array; For the first image frame Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

2. The video processing method according to claim 1, characterized in that, The method further includes: If no consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, and whose element values ​​are all greater than or equal to a preset grayscale threshold and whose quantity is greater than or equal to a second quantity threshold, then the first boundary value of the object most recently detected in the image frame preceding the first image frame will be set to... .

3. The video processing method according to claim 1, characterized in that, The method further includes: If the first image frame is the first frame image in the video, then traverse backwards starting from the last element in the first grayscale value array; If, in the first grayscale value array, a plurality of consecutive second target elements are determined to be located before the current element being traversed, adjacent to the current element being traversed, with element values ​​greater than or equal to a preset grayscale threshold, and their number greater than or equal to a second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset grayscale threshold, then a first boundary value of an object is determined to be detected in the first image frame. With the second boundary value ,in The ordinal number of the first second target element in the first grayscale value array among the consecutive plurality of second target elements. This is the ordinal number of the current element in the first grayscale array; For the first image frame Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

4. The video processing method according to claim 1, characterized in that, The step of generating a grayscale image based on the first image frame includes: Perform grayscale processing on the first image frame; The first image frame after grayscale processing is divided into multiple rectangular regions of the same size, and the average grayscale value of the pixels in each rectangular region is obtained. Traverse each pixel in the first image frame after grayscale processing. If the grayscale value of the currently traversed pixel is greater than or equal to the average value, then set the grayscale value of the currently traversed pixel to 255. If the grayscale value of the currently traversed pixel is less than the average value, then set the grayscale value of the currently traversed pixel to 0, thereby obtaining the grayscale image.

5. The video processing method according to any one of claims 1-4, characterized in that, The method further includes: Display the first image frame; The object boundary is displayed on the first image frame based on the first boundary value and the second boundary value of the new object detected in the first image frame, and the category label of the new object detected in the first image frame is displayed on the first image frame based on the position of the object boundary.

6. The video processing method according to claim 5, characterized in that, The method further includes: Acquire object tracking data, which includes a first boundary value, a second boundary value, and a category label of an object detected in image frames prior to the first image frame; The relative motion is added to the first boundary value and the second boundary value of each object in the object tracking data to obtain the updated object tracking data. The step of displaying the object boundary on the first image frame based on a first boundary value and a second boundary value of the detected new object in the first image frame, and displaying the category label of the detected new object on the first image frame based on the position of the object boundary, includes: The first boundary value, second boundary value, and category label of the new object detected in the first image frame are inserted into the updated object tracking data. On the first image frame, the boundary of the corresponding object is displayed according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and the category label of the corresponding object in the inserted object tracking data is displayed on the first image frame according to the position of the boundary of the corresponding object.

7. The video processing method according to claim 6, characterized in that, The step of displaying the boundary of the corresponding object on the first image frame based on the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data includes: Display the first image frame including the first image frame. Column pixels and the first The boundary of the rectangular image region between column pixels, where This refers to the first boundary value of the corresponding object in the inserted object tracking data. This is the second boundary value of the corresponding object in the inserted object tracking data.

8. The video processing method according to claim 6, characterized in that, Before inserting the updated object tracking data after detecting the first boundary value, second boundary value, and category label of a new object in the first image frame, the method further includes: If the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns in the first image frame, then the first boundary value, the second boundary value, and the category label of the object are removed from the updated object tracking data. The step of inserting the first boundary value, second boundary value, and category label of the new object detected in the first image frame into the updated object tracking data includes: The first boundary value, second boundary value, and category label of the new object detected in the first image frame will be inserted into the object tracking data after removal.

9. A video processing system, characterized in that, The video processing system includes: security inspection equipment, video processing equipment, and object category identification equipment, wherein: The security inspection equipment is configured to: capture video and send the video to the video processing equipment; The video processing device is configured to: acquire a first image frame in the video whose acquisition time is closest to the current time; generate a grayscale image based on the first image frame; and generate a first grayscale value array based on the grayscale image, wherein the first grayscale value array contains the first image frame whose acquisition time is closest to the current time. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. If the first image frame is not the first frame image in the video, then obtain the second grayscale value array pre-generated based on the second image frame, where the second image frame is the frame image preceding the first image frame; traverse... calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; the... Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; Obtain the first boundary value of the most recently detected object in the image frames preceding the first image frame. and from the first grayscale value array the first The first boundary value is traversed forward from the first element, wherein the detected object moves in the video from left to right. This is used to indicate the column number of the leftmost pixel in the object boundary of the most recently detected object in the second image frame; if multiple consecutive first target elements are determined in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold, then the first boundary value of the detected new object in the first image frame is obtained. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among the consecutive plurality of first target elements. The index of the current element in the first grayscale value array is used for iteration; for the first image frame... Column pixels and the first The region between columns of pixels is cropped, and the cropped image is sent to the object category identification device; The object category identification device is configured to perform object detection based on the cropped image to obtain a first category label for a new object detected in the first image frame.

10. The video processing system according to claim 9, characterized in that, The video processing device is also configured to: If no consecutive first target elements are found in the first grayscale value array that are located before the current element being traversed, adjacent to the current element being traversed, and whose element values ​​are all greater than or equal to a preset grayscale threshold and whose quantity is greater than or equal to a second quantity threshold, then the first boundary value of the object most recently detected in the image frame preceding the first image frame will be set to... .

11. The video processing system according to claim 9, characterized in that, The video processing device is also configured to: If the first image frame is the first frame image in the video, then traverse backwards starting from the last element in the first grayscale value array; If, in the first grayscale value array, a plurality of consecutive second target elements are determined to be located before the current element being traversed, adjacent to the current element being traversed, with element values ​​greater than or equal to a preset grayscale threshold, and their number greater than or equal to a second quantity threshold, and the element value of the current element being traversed is greater than or equal to the preset grayscale threshold, then a first boundary value of an object is determined to be detected in the first image frame. With the second boundary value ,in The ordinal number of the first second target element in the first grayscale value array among the consecutive plurality of second target elements. This is the ordinal number of the current element in the first grayscale array; For the first image frame Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

12. The video processing system according to claim 9, characterized in that, The step of generating a grayscale image based on the first image frame includes: Perform grayscale processing on the first image frame; The first image frame after grayscale processing is divided into multiple rectangular regions of the same size, and the average grayscale value of the pixels in each rectangular region is obtained. Traverse each pixel in the first image frame after grayscale processing. If the grayscale value of the currently traversed pixel is greater than or equal to the average value, then set the grayscale value of the currently traversed pixel to 255. If the grayscale value of the currently traversed pixel is less than the average value, then set the grayscale value of the currently traversed pixel to 0, thereby obtaining the grayscale image.

13. The video processing system according to any one of claims 9-12, characterized in that, The video processing system also includes a display device; The video processing device is also configured to: The first boundary value, the second boundary value, and the category label of the new object detected in the first image frame will be sent to the display device; The display device is configured as follows: Display the first image frame; The object boundary is displayed on the first image frame based on the first boundary value and the second boundary value of the new object detected in the first image frame, and the category label of the new object detected in the first image frame is displayed on the first image frame based on the position of the object boundary.

14. The video processing system according to claim 13, characterized in that, The video processing device is also configured to: Acquire object tracking data, which includes a first boundary value, a second boundary value, and a category label of an object detected in image frames prior to the first image frame; The relative motion is added to the first boundary value and the second boundary value of each object in the object tracking data to obtain the updated object tracking data. The step of sending the first boundary value, the second boundary value, and the category label of the new object detected in the first image frame to the display device includes: The first boundary value, second boundary value, and category label of the new object detected in the first image frame are inserted into the updated object tracking data, and the inserted object tracking data is sent to the display device. The step of displaying the object boundary on the first image frame based on a first boundary value and a second boundary value of the detected new object in the first image frame, and displaying the category label of the detected new object on the first image frame based on the position of the object boundary, includes: On the first image frame, the boundary of the corresponding object is displayed according to the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data, and the category label of the corresponding object in the inserted object tracking data is displayed on the first image frame according to the position of the boundary of the corresponding object.

15. The video processing system according to claim 14, characterized in that, The step of displaying the boundary of the corresponding object on the first image frame based on the first boundary value and the second boundary value of the corresponding object in the inserted object tracking data includes: Display the first image frame including the first image frame. Column pixels and the first The boundary of the rectangular image region between column pixels, where This refers to the first boundary value of the corresponding object in the inserted object tracking data. The second boundary value of the corresponding object in the inserted object tracking data.

16. The video processing system according to claim 14, characterized in that, Before inserting the updated object tracking data, after detecting the first boundary value, second boundary value, and category label of a new object in the first image frame, the video processing device is further configured to: If the second boundary value of any object in the updated object tracking data is greater than the maximum number of pixel columns in the first image frame, then the first boundary value, the second boundary value, and the category label of the object are removed from the updated object tracking data. The step of inserting the first boundary value, second boundary value, and category label of the new object detected in the first image frame into the updated object tracking data includes: The first boundary value, second boundary value, and category label of the new object detected in the first image frame will be inserted into the object tracking data after removal.

17. A video processing apparatus, characterized in that, include: The image frame acquisition module is configured to acquire the first image frame in the video whose capture time is closest to the current time. The first array acquisition module is configured to generate a grayscale image based on the first image frame, and generate a first grayscale value array based on the grayscale image, wherein the first grayscale value array contains the first grayscale value array. The element value of the nth element is the nth element in the grayscale image. The sum of the grayscale values ​​of the column pixels. ; The second array acquisition module is configured to acquire a second grayscale value array pre-generated based on the second image frame if the first image frame is not the first image frame in the video, wherein the second image frame is the previous image frame of the first image frame; The relative motion recognition module is set to traverse. calculate ,get Reference exercise volume ,in The first grayscale value array The element value of each element. The first grayscale value in the second grayscale array The element value of each element. The first preset quantity threshold; the... Reference exercise volume The minimum value is determined as the relative motion of the pixel between the first image frame and the second image frame. ; The first boundary acquisition module is configured to acquire the first boundary value of the most recently detected object, which is generated in advance. and from the first grayscale value array the first The elements are traversed forward, where the detected object moves in the video from left to right, and the first boundary value... Used to indicate the column number of the leftmost pixel in the object boundary of the most recently detected object in the second image frame; The second boundary acquisition module is configured to acquire the first boundary value of the detected new object in the first image frame if, in the first grayscale value array, there are a series of first target elements that are located before the current element being traversed, adjacent to the current element being traversed, have an element value greater than or equal to a preset grayscale threshold, and have a quantity greater than or equal to a second quantity threshold. With the second boundary value ,in The ordinal number of the first target element in the first grayscale value array among the consecutive plurality of first target elements. This is the ordinal number of the current element in the first grayscale array; The object detection module is configured to detect the first object in the first image frame. Column pixels and the first The region between columns of pixels is cropped, and object detection is performed based on the cropped image to obtain the category label of the new object detected in the first image frame.

18. An electronic device, characterized in that, It includes a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the video processing method according to any one of claims 1 to 8.

19. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the video processing method according to any one of claims 1 to 8.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the video processing method according to any one of claims 1 to 8.

Images