Seat belt status monitoring method
Through the video stream-based identification method, the binary video stream and color difference analysis can be used to monitor the hooking status of the seat belt in real time, solving the hidden dangers of wrong use of seat belts in high altitude operations, and achieving safety for power workers Assure.
Patent Information
- Application Number
- CN202411172049.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-26
AI Technical Summary
In high-altitude operations, when power operators wear seat belts, due to the influence of weather and physical conditions, there is a possibility of using seat belts incorrectly, resulting in safety hazards. It is difficult for the prior art to monitor and eliminate such hidden dangers in real time.
The accurate recognition method based on video stream is adopted, and the video stream collected by the camera is binarized to extract the color changes of pixel points, and the attachment status of the seat belt is determined using the color difference value and distribution characteristics to achieve real-time monitoring.
Through rapid modeling and mutation frame analysis, the processing efficiency is improved, the impact of electromagnetic field interference is reduced, real-time monitoring and analysis of the seat belt attachment status is ensured, and the risk of misjudgment and delay is reduced.
Smart Images

Figure CN118675118B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of safety detection, and more specifically to a method for monitoring the state of a seat belt. Background Art
[0002] Power workers work at heights for a long time and need to wear a suitable five-point safety belt and anchor the safety belt in a secure position. The correct wearing and use of the safety belt is an effective guarantee for the safety of workers. Under the influence of weather conditions and physical conditions, there is a possibility of safety hazards caused by incorrect use of the safety belt. Therefore, it is necessary to monitor the attachment status of the safety belt (whether it is correctly hung) in real time to eliminate safety hazards in a timely manner. Summary of the invention
[0003] In view of the above problems, the present invention provides a seat belt status monitoring method based on accurate video stream recognition.
[0004] The invention provides a method for monitoring the state of a safety belt, comprising: binarizing a video stream collected by a camera to obtain a binary video stream; calibrating the camera on the back of the safety belt with a shooting angle vertically downward; extracting a mutation frame in the binary video stream by utilizing the color change of pixel points in the binary video stream; and determining the attachment state of the safety belt by utilizing the color distribution characteristics of pixel points in the mutation frame.
[0005] According to an embodiment of the present invention, the video stream captured by the camera is binarized, including: for each pixel point of each frame of the video stream, calculating the color difference between the pixel point and the seat belt strap part; using the color difference, classifying the pixel points to obtain strap pixel points and background pixel points; setting the strap pixel points to the strap color, and setting the background pixel points to the background color; the strap color and the background color are two preset colors.
[0006] According to an embodiment of the present invention, calculating the color difference between a pixel point and a seat belt strap portion includes: extracting red, green and blue values of the seat belt strap portion using a seat belt plane image; calculating the color difference using the red, green and blue values of each pixel point and the red, green and blue values of the seat belt strap portion; wherein the calculation formula of the color difference is as follows:
[0007]
[0008] In the formula, is the red, green and blue value of the pixel; It is the red, green and blue value of the seat belt strap part.
[0009] According to an embodiment of the present invention, color difference is used to classify pixel points to obtain strap pixel points and background pixel points, including: judging the type of seat belt; the types of seat belts include monochrome seat belts and two-color seat belts; the color difference of the two-color seat belt includes the first color difference and the second color difference between the color of the pixel point and the two colors of the two-color seat belt; if the seat belt is a monochrome seat belt, judging whether the color difference of the monochrome seat belt is greater than a preset threshold; if so, the pixel point is a background pixel point; if not, the pixel point is a strap pixel point; if the seat belt is a two-color seat belt, judging whether the first color difference and the second color difference are less than the preset threshold; if the first color difference is less than the preset threshold, the pixel point is a first color pixel point; if the second color difference is less than the preset threshold, the pixel point is a second color pixel point; and determining the strap pixel points and the background pixel points using the first color pixel points and the second color pixel points.
[0010] According to an embodiment of the present invention, the first color pixel point and the second color pixel point are used to determine the hanging pixel point and the background pixel point, including: when the pixel point is neither the first color pixel point nor the second color pixel point, the pixel point is determined to be a background pixel point; the color difference between the first color and black, the color difference between the first color and white, the color difference between the second color and black, and the color difference between the second color and white are calculated; the color corresponding to the smallest color difference among the four color differences is used as the color to be analyzed, and the other color is used as the determined color; the pixel point corresponding to the color to be analyzed is the pixel point to be analyzed; the pixel point corresponding to the determined color is the hanging pixel point; for each pixel point to be analyzed, the colors of the four adjacent pixels to be analyzed are determined; if three or more of the four pixels are neither the color to be analyzed nor the determined color, the pixel point to be analyzed is a background pixel point; if the four pixels are If three or more of the pixel points are of the determined color, the pixel point to be analyzed is a hanging pixel point; if three or more of the four pixel points are of the color to be analyzed, then for each current adjacent pixel point of the color to be analyzed, the color conditions of the three adjacent pixels of the current adjacent pixel point are determined; if two or more of the three pixel points are of the determined color, the current adjacent pixel point, the pixel point to be analyzed, and the pixel points on the path between the two points are hanging pixels; if two or more of the three pixel points are neither of the color to be analyzed nor of the determined color, the current adjacent pixel point, the pixel point to be analyzed, and the pixel points on the path between the two points are background pixels; otherwise, for the three adjacent pixels of the current adjacent pixel point, each pixel point is taken as the current adjacent pixel point, and the step of determining the color conditions of the three adjacent pixels of the current adjacent pixel point is repeated.
[0011] According to an embodiment of the present invention, the color change of pixel points in the binary video stream is utilized to extract the mutation frame in the binary video stream, including: calculating the difference in the number of pixel points of the strap color between the current frame and the preset frame; the preset frame is obtained by searching the current frame forward for a preset number of frames in the binary video stream; and determining the mutation frame in the binary video stream by utilizing the difference in the number.
[0012] According to an embodiment of the present invention, a mutation frame in a binary video stream is determined by utilizing a number difference, including: determining whether the number difference is greater than a threshold; if so, determining whether the number of mutation frames is greater than a preset threshold; if the number of mutation frames is greater than the preset threshold, determining that the current frame is a mutation frame; if the number of mutation frames is less than the preset threshold, adding one to the number of mutation frames, taking the next frame of the current frame as the current frame, and repeatedly executing the step of calculating the difference in the number of pixels of the strap color between the current frame and the preset frame; if not, clearing the number of mutation frames to zero, and determining that the current frame is not a mutation frame.
[0013] According to an embodiment of the present invention, the color distribution characteristics of pixel points in the mutation frame are used to determine the attachment state of the seat belt, including: removing the mutation frame in the binary video stream, processing the binary video stream, and obtaining a fused image; using the fused image to process the mutation frame to obtain a final image, including: for each pixel point in the mutation frame whose color is the color of the strap, if the color of the pixel point in the fused image is the background color, then the color of the pixel point is changed to the background color; otherwise, the color of the pixel point remains unchanged; calculating and analyzing the number of pixel points in the final image whose color is the color of the strap; if the number of pixel points is less than a preset threshold, repeatedly executing the step of extracting the mutation frame in the binary video stream by using the color change of the pixel points in the binary video stream step; if the number of pixel points is greater than a threshold, then the distribution characteristics of all pixel points whose color is the strap color are determined; if the position distribution of all pixel points whose color is the strap color in the image is in a continuous line, then it is determined that the attachment state of the seat belt is abnormal; if the position distribution of all pixel points whose color is the strap color in the image is not in a continuous line, then a prompt signal is issued; wherein, processing the binary video stream to obtain a fused image comprises: for each pixel point, determining the number of frames in which the pixel point is the background color in the binary video stream; if the number of frames is greater than a preset threshold, setting the pixel point at the position in the fused image to the background color; if the number of frames is less than the preset threshold, setting the pixel point at the position in the fused image to the strap color.
[0014] The second aspect of the present invention provides a seat belt status monitoring device, which can be used to implement the above-mentioned seat belt status monitoring method, and the device includes: an image binarization module, which is used to binarize the video stream collected by the camera to obtain a binary video stream; the camera is calibrated on the back of the seat belt, and the shooting angle is vertically downward; a mutation frame extraction module, which is used to use the color changes of pixel points in the binary video stream to extract mutation frames in the binary video stream; a state determination module, which is used to use the color distribution characteristics of pixel points in the mutation frame to determine the attachment state of the seat belt.
[0015] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned seat belt status monitoring method.
[0016] A fourth aspect of the present invention further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned seat belt status monitoring method.
[0017] According to the seat belt status monitoring method provided by the present invention, the processing efficiency is improved by quickly building a model through a binary video stream; the seat belt status is monitored through the mutation frames in the binary video stream. Since real-time video streams are used for monitoring, the technical problem that the traditional wireless sensor monitoring method is easily interfered by electromagnetic fields in a high-altitude, high-voltage, and high-radiation environment, resulting in information judgment errors or delays, is at least partially solved, and the technical effect of real-time analysis of the seat belt attachment status is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flowchart of a seat belt status monitoring method according to an embodiment of the present invention is schematically shown;
[0019] Figure 2 A schematic diagram of a monochrome safety belt according to an embodiment of the present invention is schematically shown;
[0020] Figure 3 Schematically shows a flow chart of binarization processing of a video stream collected by a camera according to an embodiment of the present invention;
[0021] Figure 4 A flowchart of classifying pixels using color differences according to an embodiment of the present invention is schematically shown;
[0022] Figure 5 Schematically shows a flow chart of extracting a sudden change frame in a binary video stream according to an embodiment of the present invention;
[0023] Figure 6Schematically shows a flow chart for determining the attachment state of a seat belt according to an embodiment of the present invention;
[0024] Figure 7 A structural block diagram of a seat belt status monitoring device according to an embodiment of the present invention is schematically shown;
[0025] Figure 8 A block diagram of an electronic device suitable for implementing a seat belt status monitoring method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION
[0026] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.
[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0029] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0030] Figure 1 A flowchart of a seat belt status monitoring method according to an embodiment of the present invention is schematically shown. Figure 1As shown, an embodiment of the present invention provides a method for monitoring the status of a seat belt, comprising: S1, binarizing a video stream collected by a camera to obtain a binary video stream; the camera is calibrated on the back of the seat belt, and the shooting angle is vertically downward; S2, using the color change of pixels in the binary video stream to extract mutation frames in the binary video stream; S3, using the color distribution characteristics of pixels in the mutation frame to determine the attachment status of the seat belt.
[0031] In this embodiment, Figure 2 The schematic diagram of a monochrome safety belt according to an embodiment of the present invention is schematically shown. For the safety belt, the three-dimensional data of the safety belt needs to be preprocessed first. Since the safety belt for aerial work is wearable and uses a single hook or a double hook to hang the insurance, the appearance has a large consistency, so the appearance morphological features can be extracted in advance to provide data support for the subsequent accurate identification of the safety belt. The three-dimensional data preprocessing of the safety belt specifically includes: constructing a three-dimensional model of the safety belt, including the wearable part, the hanging belt part and the hook part; fitting the real color of each part of the safety belt to the surface texture of each part of the three-dimensional model; recording the real size data of the three-dimensional model.
[0032] Furthermore, if Figure 2 As shown in the figure, the camera position is calibrated on the safety belt. In order not to affect the wearing of power workers, the camera is calibrated on the safety belt. In the case of accompanying work movements, the camera calibrated on the shoulder is easily affected by the hand-raising movement, the camera calibrated on the waist is easily blocked by the legs crossing, and the camera calibrated on the chest is easily affected by the bending movement. At the same time, the camera calibrated on the back is not affected by the above work movements. Therefore, the camera is calibrated and arranged as follows Figure 2 The back position of the seat belt is shown. To ensure that it is not blocked by clothing, the camera needs to protrude 5 cm and the shooting angle is vertically downward. Before calibrating the camera, it also includes: analyzing the characteristics of high-altitude work of power workers (usually walking in small steps, often accompanied by work behaviors such as raising hands and bending over); in the three-dimensional model of the seat belt, take the three-dimensional graphic corner points such as shoulders, waist, chest, back, etc., and combine the common actions of the workers to perform visual analysis. Preferably, considering the hook angle, a front-mounted seat belt camera can be added to assist in judgment, which can achieve a more real-time and accurate seat belt hook warning.
[0033] Through the embodiments of the present invention, a binary video stream is quickly modeled and key features are quickly extracted to improve image processing efficiency; by extracting mutation frames in the binary video stream, the morphological characteristics of the seat belt hanging strap are analyzed, and then the seat belt attachment status of the power workers is analyzed in real time.
[0034] Figure 3 The flowchart of binarization processing of the video stream collected by the camera according to an embodiment of the present invention is schematically shown. Figure 3As shown, the video stream captured by the camera is binarized, including: for each pixel point of each frame of the video stream, calculating the color difference between the pixel point and the seat belt strap part; using the color difference, classifying the pixel points to obtain the strap pixel points and the background pixel points; setting the strap pixel points to the strap color, and setting the background pixel points to the background color; the strap color and the background color are two preset colors.
[0035] Through the embodiments of the present invention, by comparing the color difference between the pixel point and the seat belt, the image in the video stream is clearly divided into the strap part and the background part, thereby realizing the binarization of the video stream, which helps to reduce the noise interference in the image, improve the accuracy of the analysis, and simplify the complexity of subsequent processing.
[0036] On the basis of the above embodiment, the color difference between the pixel point and the seat belt hanging part is calculated, including: extracting the red, green and blue values of the seat belt hanging part by using the seat belt plane image; calculating the color difference by using the red, green and blue values of each pixel point and the red, green and blue values of the seat belt hanging part; wherein the calculation formula of the color difference is as follows:
[0037]
[0038] In the formula, is the red, green and blue value of the pixel; It is the red, green and blue value of the seat belt strap part.
[0039] In this embodiment, for example, taking orange (22, 07, 200) as an example, assuming that the RGB value of a certain pixel is (x, y, z), the calculation formula of the color difference is as follows:
[0040]
[0041] Through the embodiments of the present invention, the color difference calculation is performed using the red, green and blue values, so that the color difference between the pixel point and the seat belt strap part can be accurately distinguished.
[0042] Figure 4 A flowchart of classifying pixels using color differences according to an embodiment of the present invention is schematically shown. Figure 4As shown, using color difference to classify pixel points, and obtaining strap pixel points and background pixel points, includes: judging the type of seat belt; the types of seat belts include single-color seat belts and double-color seat belts; the color difference of the double-color seat belt includes the first color difference and the second color difference between the color of the pixel point and the two colors of the double-color seat belt; if the seat belt is a single-color seat belt, judging whether the color difference of the single-color seat belt is greater than a preset threshold; if so, the pixel point is a background pixel point; if not, the pixel point is a strap pixel point; if the seat belt is a double-color seat belt, judging whether the first color difference and the second color difference are less than the preset threshold; if the first color difference is less than the preset threshold, the pixel point is a first color pixel point; if the second color difference is less than the preset threshold, the pixel point is a second color pixel point; using the first color pixel point and the second color pixel point to determine the strap pixel point and the background pixel point.
[0043] In this embodiment, if the strap is a single color, the RGB value of the main color of the strap is extracted. Taking orange as an example, it is assumed to be (22, 07, 200), and the video stream is dynamically modeled in two values. The method is as follows: for a certain frame of video captured by the camera during operation, the RGB value of each pixel in the image is calculated pixel by pixel; the RGB value of each pixel is compared and calculated with orange (22, 07, 200). If the sum of the difference is greater than a preset value, such as 10%, the value of the pixel in the binary color space is defined as 0 (white). At this point, for this frame of image, the conversion to the binary color space (grayscale space) is completed: all background pixels that are orange or very close to orange are displayed as white, and other colors are displayed as black; for a continuous video stream, the above steps are repeated for each frame of image to achieve continuous binary modeling of the full time sequence image.
[0044] Through the embodiments of the present invention, the binary dynamic modeling step of the camera observation area, common modeling methods, such as Gaussian background model, mixed Gaussian model, etc., have poor real-time performance and cannot achieve rapid emergency response. The rapid modeling processing of the binary video stream is achieved through color difference; at the same time, different color difference judgment methods are adopted for seat belts of different colors, which improves the accuracy and flexibility of recognition.
[0045] According to an embodiment of the present invention, the first color pixel point and the second color pixel point are used to determine the hanging pixel point and the background pixel point, including: when the pixel point is neither the first color pixel point nor the second color pixel point, the pixel point is determined to be a background pixel point; the color difference between the first color and black, the color difference between the first color and white, the color difference between the second color and black, and the color difference between the second color and white are calculated; the color corresponding to the smallest color difference among the four color differences is used as the color to be analyzed, and the other color is used as the determined color; the pixel point corresponding to the color to be analyzed is the pixel point to be analyzed; the pixel point corresponding to the determined color is the hanging pixel point; for each pixel point to be analyzed, the colors of the four adjacent pixels to be analyzed are determined; if three or more of the four pixels are neither the color to be analyzed nor the determined color, the pixel point to be analyzed is a background pixel point; if the four pixels are If three or more of the pixel points are of the determined color, the pixel point to be analyzed is a hanging pixel point; if three or more of the four pixel points are of the color to be analyzed, then for each current adjacent pixel point of the color to be analyzed, the color conditions of the three adjacent pixels of the current adjacent pixel point are determined; if two or more of the three pixel points are of the determined color, the current adjacent pixel point, the pixel point to be analyzed, and the pixel points on the path between the two points are hanging pixels; if two or more of the three pixel points are neither of the color to be analyzed nor of the determined color, the current adjacent pixel point, the pixel point to be analyzed, and the pixel points on the path between the two points are background pixels; otherwise, for the three adjacent pixels of the current adjacent pixel point, each pixel point is taken as the current adjacent pixel point, and the step of determining the color conditions of the three adjacent pixels of the current adjacent pixel point is repeated.
[0046] In this embodiment, if the lanyard is not monochrome (at most a two-color seat belt), the method for binary dynamic modeling of the video stream is as follows: before the lanyard is put into use, its image is analyzed: a complete plane image of the lanyard is extracted; the RGB values of the two-color colors of the lanyard are extracted, and the two color combinations are recorded as color1 and color2; for a frame of video captured by the camera at work, the RGB value of each pixel in the image is calculated pixel by pixel, and all color1 and color2 areas in the image are extracted; considering that in the image, the two-color lanyard may have one color of white or black (this situation does not exist for monochrome lanyards), it is easy to be close to the RGB value of the background area in the image, so it needs to be excluded.
[0047] In this embodiment, the exclusion method is as follows: Assume that color1 is a color that is easily confused with other background areas. If a pixel is color1, let the pixel be pix0, then observe the 4 pixels adjacent to pix0 above, below, left and right, and record them as pix1, pix2, pix3 and pix4: If the colors of 3 or more pixels among pix1, pix2, pix3 and pix4 are neither color1 nor color2, it proves that the pixel is the background area; if at least 3 pixels among pix1, pix2, pix3 and pix4 are color2, then the pixel pix0 is determined to be the strap area; if at least 3 pixels among pix1, pix2, pix3 and pix4 are color1, further judgment is required. The process of further judgment is as follows: assume that the three pixels of color1 are pix1, pix2 and pix3; continue to judge the colors of the upper, lower, left and right adjacent pixels of pix1, pix2 and pix3, with the same method as above. If at least 2 (one direction is pix0) are color2, then the pixel and pix0 are considered to be in the strap area; otherwise, continue to observe the adjacent upper, lower, left and right pixels until a pixel satisfies "at least 2 of the adjacent pixels are neither color1 nor color2". All iteratively connected pixels of color1 are marked as the background area; pixels whose color is color1 (strap color) but are in the background area are marked as black in the binary space.
[0048] Through the embodiments of the present invention, by performing detailed analysis and processing on edge pixels in particular, the recognition accuracy of image edges is improved. At the same time, through multiple iterations and color judgment of adjacent pixels, the influence of noise on the recognition results can be reduced, and the color of the pixels of the two-color seat belts and their adjacent pixels is taken into consideration to more accurately judge the ownership of the pixels (sling or background).
[0049] Figure 5 The flowchart of extracting a sudden change frame in a binary video stream according to an embodiment of the present invention is schematically shown. Figure 5 As shown, the color change of pixels in a binary video stream is used to extract mutation frames in the binary video stream, including: calculating the difference in the number of pixels with the color of the strap between the current frame and the preset frame; the preset frame is obtained by searching the current frame forward for a preset number of frames in the binary video stream; and determining the mutation frame in the binary video stream using the difference in the number.
[0050] In this embodiment, in the step of extracting sudden frames from the video stream, the operator is calibrated on the back, and the camera with a vertical downward shooting angle is mainly used to observe the state of the sling part to ensure whether the hook is towed (for hanging). The principle is: when the sling part is towed, it will fall vertically downward, and its shape is vertically downward or in an arc downward (in windy conditions, and the weight of the hook part is much larger than the sling part). Both states will be captured by the camera (the camera is calibrated to protrude 5 cm). Therefore, the key frame extraction can use "sudden appearance (increase) of the sling part" to anchor the sudden frame.
[0051] In this embodiment, the specific implementation principle of identifying the "sudden appearance (increase) of the hanging strap portion" is: assuming that the sampling frequency of the camera is f frames / second, the time interval between two adjacent frames of images is 1 / f seconds; when the hook falls off, it is a free fall, and the moving distance is calculated by the acceleration displacement formula. When the video sampling frequency f>15 frames / second, the hanging strap moves a distance of at most 0.02 meters within 1 / f seconds between two frames of images; 0.02 meters is very small in image size, which is not conducive to judgment, so the "binary change" between two adjacent frames of images cannot be judged to extract the mutation frame (used to judge whether the hook has fallen off). Therefore, the present invention adopts the method of "image frame interval comparison" to solve the above problem. The specific steps are: calculate the number of pixels with a value of 1 (black) in the binary space of the current frame image; compare it with the number of pixels with a value of 1 in the 10th frame image before the current frame (time interval 10 / f seconds); if the difference in the number of pixels is greater than a threshold value (for example, 50), continue to compare the next frame image with the 9th frame image before the current frame (the time interval is still 10 / f seconds); repeat the above three times. If the results of the three comparisons show that the difference in the number of pixels with a value of 1 is greater than the threshold value, the current frame is defined as a "mutation frame".
[0052] Through the embodiments of the present invention, by monitoring the color changes of pixels in the video stream, the dynamic changes of the seat belt can be captured in real time; by calculating the difference in the number of strap color pixels in the current frame and the preset frame, possible mutation frames can be quickly screened out.
[0053] On the basis of the above embodiment, the number difference is used to determine the mutation frame in the binary video stream, including: judging whether the number difference is greater than a threshold; if so, judging whether the number of mutation frames is greater than a preset threshold; if the number of mutation frames is greater than the preset threshold, determining that the current frame is a mutation frame; if the number of mutation frames is less than the preset threshold, adding one to the number of mutation frames, taking the next frame of the current frame as the current frame, and repeating the step of calculating the difference in the number of pixels of the strap color between the current frame and the preset frame; if not, clearing the number of mutation frames to zero, and determining that the current frame is not a mutation frame.
[0054] Through the embodiments of the present invention, by judging the number of mutation frames and the preset threshold value of the number of mutation frames, misjudgment caused by accidental factors is reduced, and the accuracy of mutation frame extraction is improved.
[0055] Figure 6 A flowchart for determining the attachment state of a seat belt according to an embodiment of the present invention is schematically shown. Figure 6 As shown, the color distribution characteristics of the pixel points in the mutation frame are used to determine the hanging state of the seat belt, including: removing the mutation frame in the binary video stream, processing the binary video stream, and obtaining a fused image; using the fused image to process the mutation frame to obtain the final image, including: for each pixel point in the mutation frame whose color is the color of the strap, if the color of the pixel point in the fused image is the background color, then the color of the pixel point is changed to the background color; otherwise, the color of the pixel point remains unchanged; calculating and analyzing the number of pixels whose color is the color of the strap in the final image; if the number of pixels is less than a preset threshold, then repeatedly executing the step of extracting the mutation frame in the binary video stream by using the color change of the pixel points in the binary video stream ... If the number of pixels is greater than a threshold, the distribution characteristics of all pixels whose color is the strap color are determined; if the position distribution of all pixels whose color is the strap color in the image is in a continuous line, it is determined that the attachment state of the seat belt is abnormal; if the position distribution of all pixels whose color is the strap color in the image is not in a continuous line, a prompt signal is issued; wherein, processing the binary video stream to obtain a fused image comprises: for each pixel, determining the number of frames in which the pixel is the background color in the binary video stream; if the number of frames is greater than a preset threshold, setting the pixel at the position in the fused image to the background color; if the number of frames is less than the preset threshold, setting the pixel at the position in the fused image to the strap color.
[0056] In this embodiment, the current frame image is removed from the n binary images of the binary video stream to obtain n-1 binary images; the n-1 binary images are fused pixel by pixel, and the fusion method is: if a pixel point has a value of 1 (black, background part) in more than 10 frames of images, the pixel value of the fused image at that position is set to 1; in other cases, the pixel value is set to 0; finally, the fused image is obtained; the area where the pixel point in the image of the current frame is 1 is calculated, and if the pixel point is 0 in the "fused image", the point value is changed to 0, otherwise it is 1, and the "final image" after the current frame is processed is obtained. Explanatory note, this step is to solve the problem that the bottom of the strap (the strap near the camera) may "accidentally enter" the camera shooting area due to the body movement of the operator.
[0057] In this embodiment, the number of pixels with a value of 1 in the "final image" is calculated and analyzed. If it is less than a threshold value (such as 30), the mutation frame is determined to be abnormal, and the mutation frame of the subsequent video is re-extracted; if it is greater than the threshold value, the positions of all pixels with a value of 1 in the image are further calculated; if the pixel distribution calculated from the pixel position is continuous and the contour is close to a "linear" state, it is determined to be "hook detached" and the early warning and emergency treatment plan is entered; if the pixel calculated from the pixel position has no obvious distribution characteristics, the operator is prompted to check the hook.
[0058] Through the embodiments of the present invention, by fusing images and further processing mutation frames, the pixels of the lanyard that have mistakenly entered the camera shooting area are repaired as background pixels, thereby improving the judgment accuracy of the mutation frames and facilitating pixel distribution analysis; the attachment status of the seat belt can be accurately judged through the pixel distribution characteristics.
[0059] Based on the above seat belt status monitoring device, the present invention also provides a seat belt status monitoring device, which can be used to implement the above seat belt status monitoring method. Figure 7 The device is described in detail.
[0060] like Figure 7 As shown, the seat belt status monitoring device 700 of this embodiment includes an image binarization module 701 , a mutation frame extraction module 702 and a status determination module 703 .
[0061] The image binarization module 701 is used to binarize the video stream collected by the camera to obtain a binary video stream; the camera is calibrated on the back of the seat belt, and the shooting angle is vertically downward. In one embodiment, the image binarization module 701 can be used to perform the operation S1 described above, which will not be repeated here.
[0062] The sudden frame extraction module 702 is used to extract the sudden frame in the binary video stream by utilizing the color change of the pixel points in the binary video stream. In one embodiment, the sudden frame extraction module 702 can be used to perform the operation S2 described above, which will not be described in detail here.
[0063] The state determination module 703 is used to determine the attachment state of the safety belt by using the color distribution characteristics of the pixels in the mutation frame. In one embodiment, the state determination module 703 can be used to perform the operation S3 described above, which will not be described in detail here.
[0064] Figure 8 A block diagram of an electronic device suitable for implementing a seat belt status monitoring method according to an embodiment of the present invention is schematically shown.
[0065] like Figure 8As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.
[0066] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.
[0067] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.
[0068] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.
[0069] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.
[0070] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0071] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0072] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.
[0073] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.
Claims
1. A seat belt status monitoring method, characterized in that: include: Binarize the video stream collected by the camera to obtain a binary video stream; The camera is calibrated on the back of the seat belt, with the shooting angle vertically downward; The binarization process of the video stream collected by the camera includes: for each pixel point of each frame of the video stream, calculating the color difference between the pixel point and the seat belt strap part; using the color difference, classifying the pixel points to obtain strap pixel points and background pixel points; setting the strap pixel points to the strap color, and setting the background pixel points to the background color; the strap color and the background color are two preset colors; Extracting a sudden change frame in the binary video stream by utilizing the color change of the pixel points in the binary video stream; Determine the attachment state of the seat belt by using the color distribution characteristics of the pixel points in the mutation frame, specifically including: remove the mutation frame in the binary video stream, process the binary video stream, and obtain a fused image; use the fused image to process the mutation frame to obtain a final image, including: for each pixel point in the mutation frame whose color is the color of the strap, if the color of the pixel point in the fused image is the background color, then change the color of the pixel point to the background color; otherwise, the color of the pixel point remains unchanged; calculate and analyze the number of pixels whose color is the color of the strap in the final image; if the number of pixels is less than a preset threshold, repeat the step of extracting the mutation frame in the binary video stream by using the color change of the pixel points in the binary video stream; if the number of pixels is greater than the threshold, determine the distribution characteristics of all pixels whose color is the color of the strap; if the position distribution of all pixels whose color is the color of the strap in the image is continuous line, it is determined that the attachment state of the seat belt is abnormal; if the position distribution of all pixels whose color is the color of the strap in the image is not continuous line, then send a prompt signal; The processing of the binary video stream to obtain a fused image includes: For each pixel, determine the number of frames in which the pixel is the background color in the binary video stream; if the number of frames is greater than a preset threshold, set the pixel at that position in the fused image to the background color; if the number of frames is less than the preset threshold, set the pixel at that position in the fused image to the strap color.
2. The method according to claim 1, wherein: Calculate the color difference between the pixel and the seat belt strap, including: The red, green and blue values of the seat belt strap part are extracted using the seat belt plane image; Calculate the color difference using the red, green, and blue values of each pixel and the red, green, and blue values of the seat belt strap. The calculation formula of the color difference is as follows: ; In the formula, is the red, green and blue value of the pixel; It is the red, green and blue value of the seat belt strap part.
3. The method according to claim 1, wherein: Using the color difference, classifying the pixels to obtain the strap pixels and background pixels, including: Determine the type of the seat belt; the type of the seat belt includes a single-color seat belt and a double-color seat belt; the color difference of the double-color seat belt includes a first color difference and a second color difference between the color of the pixel point and the two colors of the double-color seat belt; If the safety belt is a monochrome safety belt, determine whether the color difference of the monochrome safety belt is greater than a preset threshold; if so, the pixel point is a background pixel point; if not, the pixel point is a strap pixel point; If the seat belt is a two-color seat belt, determine whether the first color difference and the second color difference are less than a preset threshold; if the first color difference is less than the preset threshold, the pixel point is a first color pixel point; if the second color difference is less than the preset threshold, the pixel point is a second color pixel point; use the first color pixel point and the second color pixel point to determine the strap pixel point and the background pixel point.
4. The method according to claim 3, wherein: Determining strap pixels and background pixels using the first color pixels and the second color pixels includes: When the pixel point is neither a first color pixel point nor a second color pixel point, determining the pixel point as a background pixel point; Calculate the color difference between the first color and black, the color difference between the first color and white, the color difference between the second color and black, and the color difference between the second color and white; The color corresponding to the smallest color difference among the four color differences is used as the color to be analyzed, and the other color is used as the determined color; the pixel point corresponding to the color to be analyzed is the pixel point to be analyzed; the pixel point corresponding to the determined color is the hanging band pixel point; For each pixel to be analyzed, determine the colors of four pixels adjacent to the pixel to be analyzed; If three or more of the four pixels are neither the color to be analyzed nor the determined color, the pixel to be analyzed is a background pixel; If three or more of the four pixels are of a certain color, the pixel to be analyzed is a hanging band pixel; If three or more of the four pixels are the color to be analyzed, then for each current adjacent pixel of the color to be analyzed, the color of the three adjacent pixels of the current adjacent pixel is determined; if two or more of the three pixels are the determined color, the current adjacent pixel, the pixel to be analyzed and the pixels on the path between the two points are hanging pixels; if two or more of the three pixels are neither the color to be analyzed nor the determined color, the current adjacent pixel, the pixel to be analyzed and the pixels on the path between the two points are background pixels; otherwise, for the three adjacent pixels of the current adjacent pixel, take each pixel as the current adjacent pixel and repeat the step of determining the color of the three adjacent pixels of the current adjacent pixel.
5. The method according to claim 1, wherein: Extracting a sudden change frame in the binary video stream by utilizing the color change of the pixel points in the binary video stream, comprising: Calculating the difference in the number of pixels of the strap color between the current frame and the preset frame; the preset frame is obtained by searching the current frame forward for a preset number of frames in the binary video stream; The mutation frames in the binary video stream are determined by using the number difference.
6. The method according to claim 5, wherein: Determining the sudden change frame in the binary video stream by using the number difference includes: Determine whether the number difference is greater than a threshold; If so, determine whether the number of mutation frames is greater than a preset threshold; if the number of mutation frames is greater than the preset threshold, determine that the current frame is a mutation frame; if the number of mutation frames is less than the preset threshold, add one to the number of mutation frames, take the next frame of the current frame as the current frame, and repeat the step of calculating the difference in the number of pixels of the strap color between the current frame and the preset frame; If not, the mutation frame number is cleared to determine that the current frame is not a mutation frame.
7. A seat belt status monitoring device, characterized in that: The device can be used to implement the method according to any one of claims 1 to 6, and the device comprises: The image binarization module is used for binarizing the video stream collected by the camera to obtain a binarized video stream; the camera is calibrated on the back of the seat belt, and the shooting angle is vertically downward; the binarization of the video stream collected by the camera includes: for each pixel point of each frame of the video stream, calculating the color difference between the pixel point and the seat belt hanging strap part; using the color difference, classifying the pixel points to obtain the hanging strap pixel points and the background pixel points; setting the hanging strap pixel points to the hanging strap color, and setting the background pixel points to the background color; the hanging strap color and the background color are two preset colors; A mutation frame extraction module, used to extract mutation frames in the binary video stream by utilizing the color changes of pixels in the binary video stream; The state determination module is used to determine the attachment state of the safety belt by using the color distribution characteristics of the pixels in the mutation frame, specifically including: removing the mutation frame in the binary video stream, processing the binary video stream, and obtaining a fused image; using the fused image to process the mutation frame to obtain a final image, including: for each pixel in the mutation frame whose color is the color of the strap, if the color of the pixel in the fused image is the background color, then changing the color of the pixel to the background color; otherwise, the color of the pixel remains unchanged; calculating and analyzing the number of pixels whose color is the color of the strap in the final image; if the number of pixels is less than a preset threshold, repeatedly executing the step of extracting the mutation frame in the binary video stream by using the color change of the pixels in the binary video stream; if the number of pixels is greater than the threshold, determining the distribution characteristics of all pixels whose color is the color of the strap; if the position distribution of all pixels whose color is the color of the strap in the image is continuous and linear, determining that the attachment state of the safety belt is abnormal; if the position distribution of all pixels whose color is the color of the strap in the image is not continuous and linear, issuing a prompt signal; The processing of the binary video stream to obtain a fused image includes: For each pixel, determine the number of frames in which the pixel is the background color in the binary video stream; if the number of frames is greater than a preset threshold, set the pixel at that position in the fused image to the background color; if the number of frames is less than the preset threshold, set the pixel at that position in the fused image to the strap color.
8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Protection equipment wearing detection method and device based on unmanned aerial vehicle vision
CN112307913A
High-altitude operation safety belt detection early warning method and system
CN117437442A
A system for recognizing workers' wearing safety protection equipment through real-time image analysis
KR1020220051609A
Iterative recognition-guided thresholding and data extraction
US20170024629A1
Method and system for seatbelt detection using adaptive histogram normalization
US20230196798A1
