A monitoring and recognition method and device based on machine vision

Abstract

This invention discloses a monitoring and recognition method and apparatus based on machine vision, relating to the field of machine vision technology. The method includes: acquiring an image of a target region and extracting pixel values ​​corresponding to all pixels; classifying pixels into different states based on pixel values ​​and preset pixel thresholds, including strong pixels, weak pixels, and normal pixels; dynamically adjusting corresponding learning weight coefficients based on the different states of pixels; performing image enhancement operations based on the learning weight coefficients to obtain a target image; substituting the target image into a target recognition model to obtain a recognition result; the recognition result indicating whether behavioral characteristics of a target object exist within the target region; and determining the abnormal behavioral characteristics of the target object based on the recognition result of the target image. This invention improves image visibility and then performs precise target object recognition, increasing the accuracy of target object recognition and thus reducing security human resource costs.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a monitoring and recognition method and apparatus based on machine vision. Background Technology

[0002] In machine vision systems, the processing and analysis of optical information captured by front-end imaging devices (such as industrial cameras, lenses, and light sources) ensures that this image data can be efficiently transformed into digital information usable for back-end decision-making. Machine vision systems utilize a series of image acquisition and processing facilities, such as image acquisition cards, GPU computing units, and embedded controllers, to compress high-dimensional raw images and convert them into multi-scale features suitable for algorithm analysis. They then construct multi-layered networks to provide real-time and reliable visual perception services to various industrial, security, and transportation scenarios. This provides a stable and accurate image recognition foundation for applications such as intelligent manufacturing, security protection, and autonomous driving.

[0003] Existing technology (publication number: CN119992476A) provides a method and device for obstacle monitoring in rail transit stations based on machine vision, relating to the fields of machine vision and rail transit technology. The method includes: inputting a target image generated from surveillance video into a lightweight deep learning model to predict and generate obstacle recognition results; then, based on the obstacle recognition results, using a multi-survey video fusion algorithm to verify and determine the target obstacle. This multi-survey video fusion algorithm is used to match the identified obstacle with surveillance videos from different video sources within the rail transit station and to remove duplicate obstacles. This method solves the technical problems of low efficiency and low accuracy in obstacle recognition in existing technologies, achieving the technical effect of improving the accuracy and reliability of obstacle recognition in stations.

[0004] However, in practical applications, when a target object enters a factory building from the outside in an instant, there will be a phenomenon of exposure changing from strong to weak. This makes it difficult for the camera equipment to capture both the bright and dark details of the target object's behavioral characteristics at the same time, reducing the accuracy of target object identification. Summary of the Invention

[0005] The purpose of this invention is to solve the problem by proposing a monitoring and identification method and device based on machine vision.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] First, a monitoring and recognition method based on machine vision is proposed, the method comprising:

[0008] Acquire an image of the target region, acquire all pixels of the image, and extract the pixel values ​​corresponding to all pixels;

[0009] Pixels in different states are obtained by dividing them according to their pixel values ​​and preset pixel thresholds. The pixels in different states include strong pixels, weak pixels, and normal pixels.

[0010] The learning weight coefficients are dynamically adjusted based on the pixel's state.

[0011] The target image is obtained by performing enhancement operations on the image based on the learned weight coefficients.

[0012] The target image is substituted into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, the abnormal behavioral characteristics of the target object are determined.

[0013] Optionally, pixels in different states can be divided based on their pixel values ​​and preset pixel thresholds, including:

[0014] The preset pixel threshold includes a lower limit threshold and an upper limit threshold for preset pixels;

[0015] If a pixel value is greater than or equal to the upper threshold, then the pixel is a strong pixel.

[0016] If the lower threshold is less than the pixel value and the upper threshold is less than the upper threshold, then the pixel is a normal pixel.

[0017] If the pixel value is less than or equal to the lower threshold, then the pixel is a weak pixel.

[0018] Optionally, the learning weight coefficients that are dynamically adjusted according to the pixel in different states include:

[0019] The average strong pixel value is obtained by averaging the pixel values ​​corresponding to all strong pixels. The average strong pixel value is then substituted into the target training neural network model to obtain the first learning weight coefficient.

[0020] The average weak pixel value is obtained by averaging the pixel values ​​corresponding to all weak pixels. The average weak pixel value is then substituted into the target training neural network model to obtain the second learning weight coefficient.

[0021] The learning weight coefficients include a first learning weight coefficient and a second learning weight coefficient.

[0022] Optionally, performing image enhancement operations based on the learned weight coefficients to obtain the target image includes:

[0023] S1: The formula for calculating the enhancement operation is:

[0024]

[0025] Where LS(x) represents the pixel value output after image adjustment, Y(x) represents the normalized input pixel value, x represents the pixel point, ρ represents the learning weight coefficient, and LS n (x) represents the pixel value output after the nth enhancement, where n represents the total number of enhancements;

[0026] S2: Perform step S1 on all strong pixels and all weak pixels; when the number of iterations reaches n, output the enhanced image, which is denoted as the target image.

[0027] Optionally, the target image can be substituted into the target recognition model to obtain the recognition result, including:

[0028] The target image is used as the input to the target recognition model; the target image is subjected to continuous downsampling operations to obtain the corresponding first sampled image, second sampled image and third sampled image respectively;

[0029] The third sampled image is fused to obtain the first fused feature image;

[0030] The first fused feature image is upsampled and then fused with the second sampled image to obtain the second fused feature image.

[0031] After upsampling the second fused feature image, it is fused with the first sampled image to obtain the third fused feature image;

[0032] The first fused feature image, the second fused feature image, and the third fused feature image are respectively connected to the corresponding first detection head, second detection head, and third detection head, and the recognition result is obtained based on the output features of each detection head.

[0033] A machine vision-based monitoring and recognition device is proposed, comprising:

[0034] Image acquisition module: Acquires an image of the target area, acquires all pixels of the image, and extracts the pixel values ​​corresponding to all pixels;

[0035] Pixel segmentation module: Divides pixels into different states based on pixel values ​​and preset pixel thresholds. The different states of pixels include strong pixels, weak pixels, and normal pixels.

[0036] Image enhancement module: dynamically adjusts the corresponding learning weight coefficients according to pixels in different states; performs image enhancement operations based on the learning weight coefficients to obtain the target image;

[0037] Image recognition module: Substitutes the target image into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, the abnormal behavioral characteristics of the target object are determined.

[0038] Optionally, the pixel segmentation module includes: a threshold segmentation module, a first judgment module, a second judgment module, and a third judgment module.

[0039] The threshold division module includes a preset pixel threshold, which comprises a lower limit threshold and an upper limit threshold for preset pixels.

[0040] The first judgment module is used to determine if a pixel is a strong pixel if the pixel value is greater than or equal to the upper limit threshold.

[0041] The second judgment module is used to determine if a pixel is a normal pixel if the lower threshold is less than the pixel value and the upper threshold is less than the upper threshold.

[0042] The third judgment module is used to determine if a pixel is a weak pixel if its value is less than or equal to the lower threshold.

[0043] Optionally, the image enhancement module includes: a first weight module, a second weight module, and a learned weight partitioning module.

[0044] The first weighting module is used to calculate the average value of the pixel values ​​corresponding to all strong pixels to obtain the average strong pixel value, and to substitute the average strong pixel value into the target training neural network model to obtain the first learning weight coefficient.

[0045] The first weighting module is used to calculate the average weak pixel value by averaging the pixel values ​​corresponding to all weak pixels, and to substitute the average weak pixel value into the target training neural network model to obtain the second learning weight coefficient.

[0046] The learning weight partitioning module includes learning weight coefficients, including a first learning weight coefficient and a second learning weight coefficient.

[0047] Optionally, performing image enhancement operations based on the learned weight coefficients to obtain the target image includes:

[0048] S1: The formula for calculating the enhancement operation is:

[0049]

[0050] Where LS(x) represents the pixel value output after image adjustment, Y(x) represents the normalized input pixel value, x represents the pixel point, ρ represents the learning weight coefficient, and LS n (x) represents the pixel value output after the nth enhancement, where n represents the total number of enhancements;

[0051] S2: Perform step S1 on all strong pixels and all weak pixels; when the number of iterations reaches n, output the enhanced image, which is denoted as the target image.

[0052] Optionally, the image recognition module includes: a downsampling module, a first fusion module, a second fusion module, a third fusion module, and a detection head module.

[0053] The downsampling module is used to take the target image as input to the target recognition model; and to perform continuous downsampling operations on the target image to obtain the corresponding first sampled image, second sampled image and third sampled image respectively.

[0054] The first fusion module is used to perform a fusion operation on the third sampled image to obtain a first fused feature image;

[0055] The second fusion module is used to perform an upsampling operation on the first fusion feature image and then perform a fusion operation with the second sampled image to obtain a second fusion feature image;

[0056] The third fusion module is used to perform an upsampling operation on the second fusion feature image and then perform a fusion operation with the first sampled image to obtain a third fusion feature image.

[0057] Detection head module: Connects the first fused feature image, the second fused feature image, and the third fused feature image to the corresponding first detection head, second detection head, and third detection head, respectively, and obtains the recognition result based on the output features of each detection head.

[0058] The beneficial effects of this invention are:

[0059] This invention proposes a machine vision-based surveillance and recognition method. It acquires an image of a target area, obtains all pixels in the image, and extracts the pixel values ​​corresponding to each pixel. Pixels are then categorized into different states—strong, weak, and normal—based on their pixel values ​​and preset pixel thresholds. Learning weight coefficients are dynamically adjusted according to these different states. The image is then enhanced using these learning weight coefficients to obtain the target image. This target image is then fed into a target recognition model to obtain the recognition result. The recognition result indicates whether behavioral characteristics of a target object exist within the target area. Based on the recognition result of the target image, abnormal behavioral characteristics of the target object are determined. This invention improves image visibility by enhancing low-light images and suppressing high-exposure images, thereby enhancing the accuracy of target object recognition and reducing security human resource costs. Attached Figure Description

[0060] Figure 1A flowchart illustrating a machine vision-based monitoring and recognition method provided in an embodiment of the present invention;

[0061] Figure 2 A flowchart of a target recognition model provided in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of a machine vision-based monitoring and recognition device provided in an embodiment of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] This invention provides a machine vision-based monitoring and recognition method. See also... Figure 1 , Figure 1 A flowchart illustrating a machine vision-based monitoring and recognition method provided in an embodiment of the present invention. The method includes the following steps:

[0065] Acquire the image of the target region, obtain all pixels in the image, and extract the pixel values ​​corresponding to all pixels;

[0066] Pixels are divided into different states based on their pixel values ​​and preset pixel thresholds. Pixels in different states include strong pixels, weak pixels, and normal pixels.

[0067] The learning weight coefficients are dynamically adjusted based on the pixel's state.

[0068] The target image is obtained by performing enhancement operations on the image based on the learned weight coefficients;

[0069] The target image is substituted into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, the abnormal behavioral characteristics of the target object are determined.

[0070] Specifically, the target areas are factories, industrial parks, etc.; the target objects are such as vehicle license plates, and the behavioral characteristics of people, including facial features and human posture.

[0071] The machine vision-based monitoring and identification method provided by this invention enhances low-light images and suppresses high-exposure images to obtain output images, thereby improving image visibility. Then, it accurately identifies target objects, improving the accuracy of target object identification and reducing security human resource costs.

[0072] In one implementation, pixels are divided into different states based on their pixel values ​​and a preset pixel threshold, including:

[0073] The preset pixel thresholds include the lower limit threshold and the upper limit threshold of preset pixels;

[0074] If a pixel value is greater than or equal to the upper threshold, then the pixel is a strong pixel.

[0075] If the lower threshold is less than the pixel value and the upper threshold is less than the upper threshold, then the pixel is a normal pixel.

[0076] If the pixel value is less than or equal to the lower threshold, then the pixel is a weak pixel.

[0077] In one implementation, the learning weight coefficients are dynamically adjusted according to the pixel states, including:

[0078] The average strong pixel value is obtained by averaging the pixel values ​​corresponding to all strong pixels. The average strong pixel value is then substituted into the target training neural network model to obtain the first learning weight coefficient.

[0079] The average weak pixel value is obtained by averaging the pixel values ​​corresponding to all weak pixels. The average weak pixel value is then substituted into the target training neural network model to obtain the second learning weight coefficient.

[0080] The learning weight coefficients include the first learning weight coefficient and the second learning weight coefficient.

[0081] In one implementation, the specific training process of the target training neural network model includes:

[0082] The process involves retrieving individual pixels and their corresponding learning weight coefficients from a database, integrating these data into training and testing data, importing the training data into an AI model, and then using the testing data to evaluate the trained AI model. The final result is a target neural network model with each pixel and its corresponding learning weight coefficient as input and the learning weight coefficients as output. AI models include CNN models, among others.

[0083] In one implementation, the first learning weight coefficient has a value range of (-1, 0], which has a suppressing (darkening) effect on the image, and the second learning weight coefficient has a value range of (0, 1), which has an enhancing (brightening) effect on the image.

[0084] In one implementation, the image enhancement operation based on the learned weight coefficients to obtain the target image includes:

[0085] S1: The formula for calculating the enhancement operation is:

[0086]

[0087] Where LS(x) represents the pixel value output after image adjustment, Y(x) represents the normalized input pixel value, x represents the pixel point, ρ represents the learning weight coefficient, and LS n (x) represents the pixel value output after the nth enhancement, where n represents the total number of enhancements;

[0088] S2: Perform step S1 on all strong pixels and all weak pixels; when the number of iterations reaches n, output the enhanced image, which is denoted as the target image.

[0089] In one implementation, the image is enhanced iteratively to achieve finer pixel-level enhancement, improving the visual effect of the generated image by enhancing the low-light image pixel by pixel.

[0090] See Figure 2 , Figure 2 A flowchart of a target recognition model provided in an embodiment of the present invention is provided. In one implementation, substituting the target image into the target recognition model to obtain the recognition result includes:

[0091] The target image is used as the input to the target recognition model; the target image is continuously downsampled to obtain the corresponding first sampled image, second sampled image and third sampled image respectively;

[0092] The third sampled image is fused to obtain the first fused feature image;

[0093] The first fused feature image is obtained by upsampling the first fused feature image and then fusing it with the second sampled image.

[0094] The second fused feature image is upsampled and then fused with the first sampled image to obtain the third fused feature image.

[0095] The first fused feature image, the second fused feature image, and the third fused feature image are respectively connected to the corresponding first detection head, second detection head, and third detection head, and the recognition result is obtained based on the output features of each detection head.

[0096] In one implementation, multi-scale feature maps are generated through three consecutive levels of downsampling. High-resolution details are then progressively superimposed using a bottom-up fusion path, ensuring the same target retains complete semantics at both extremely small and extremely large scales. Three matching detection heads output in parallel, capturing both large-scale vehicle trajectories and accurately locating localized abnormal human movements. In factory monitoring scenarios, even with tilted camera angles, drastic lighting changes, or partial obstruction of the target by equipment, the system can still identify abnormal behaviors such as boundary violations, loitering, not wearing safety helmets, and vehicles driving in the wrong direction within milliseconds. This significantly reduces security manpower and operational costs, providing a highly reliable, low-latency proactive early warning barrier for safe production.

[0097] In one implementation, the computational process for multi-scale feature fusion includes:

[0098]

[0099] Where A0 represents the input feature of the multi-scale feature fusion, A1 represents the convolutional feature of the input feature, A2, A3, and A4 are the outputs of the three convolutional branches after average pooling, respectively; A5 is the convolution weight; A6 is the output of the multi-scale feature enhancement module; F is the operator, the subscript Conv represents the standard convolution, and the superscript represents the kernel size. This represents the cross product symbol.

[0100] Based on the same inventive concept, this invention also provides a machine vision-based monitoring and identification device. See also Figure 3 , Figure 3 A schematic diagram of a machine vision-based monitoring and recognition device provided in an embodiment of the present invention includes:

[0101] Image acquisition module: Acquires the image of the target area, acquires all pixels of the image, and extracts the pixel values ​​corresponding to all pixels;

[0102] Pixel segmentation module: Divides pixels into different states based on pixel values ​​and preset pixel thresholds. Pixels in different states include strong pixels, weak pixels, and normal pixels.

[0103] Image enhancement module: dynamically adjusts the corresponding learning weight coefficients according to pixels in different states; performs image enhancement operations based on the learning weight coefficients to obtain the target image;

[0104] Image recognition module: Substitutes the target image into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, the abnormal behavioral characteristics of the target object are determined.

[0105] The machine vision-based monitoring and recognition device provided in this invention enhances low-light images and suppresses high-exposure images to obtain output images, thereby improving image visibility. Then, it accurately identifies target objects, improving the accuracy of target object identification and reducing security human resource costs.

[0106] It should be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0107] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A monitoring and recognition method based on machine vision, characterized in that, The method includes: Acquire an image of the target region, acquire all pixels of the image, and extract the pixel values ​​corresponding to all pixels; Pixels in different states are obtained by dividing them according to their pixel values ​​and preset pixel thresholds. The pixels in different states include strong pixels, weak pixels, and normal pixels. The learning weight coefficients are dynamically adjusted based on the pixel's state. The target image is obtained by performing enhancement operations on the image based on the learned weight coefficients; The target image is substituted into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, the abnormal behavioral characteristics of the target object are determined. The learning weight coefficients that are dynamically adjusted according to pixels in different states include: The average strong pixel value is obtained by averaging the pixel values ​​corresponding to all strong pixels. The average strong pixel value is then substituted into the target training neural network model to obtain the first learning weight coefficient. The average weak pixel value is obtained by averaging the pixel values ​​corresponding to all weak pixels. The average weak pixel value is then substituted into the target training neural network model to obtain the second learning weight coefficient. The learning weight coefficients include a first learning weight coefficient and a second learning weight coefficient; The step of performing image enhancement operations based on the learned weight coefficients to obtain the target image includes: S1: The formula for calculating the enhancement operation is: Where LS(x) represents the pixel value output after image adjustment, Y(x) represents the normalized input pixel value, x represents the pixel point, and ρ represents the learning weight coefficient. This represents the pixel value output after the nth enhancement, where n represents the total number of enhancements. S2: Perform step S1 on all strong pixels and all weak pixels; when the number of iterations reaches n, output the enhanced image, which is denoted as the target image.

2. The monitoring and recognition method based on machine vision according to claim 1, characterized in that, The process of dividing pixels into different states based on pixel values ​​and preset pixel thresholds includes: The preset pixel threshold includes a lower limit threshold and an upper limit threshold for preset pixels; If a pixel value is greater than or equal to the upper threshold, then the pixel is a strong pixel. If the lower threshold is less than the pixel value and the upper threshold is less than the upper threshold, then the pixel is a normal pixel. If the pixel value is less than or equal to the lower threshold, then the pixel is a weak pixel.

3. The monitoring and recognition method based on machine vision according to claim 1, characterized in that, The step of substituting the target image into the target recognition model to obtain the recognition result includes: The target image is used as the input to the target recognition model; the target image is subjected to continuous downsampling operations to obtain the corresponding first sampled image, second sampled image and third sampled image respectively; The third sampled image is fused to obtain the first fused feature image; The first fused feature image is upsampled and then fused with the second sampled image to obtain the second fused feature image. After upsampling the second fused feature image, it is fused with the first sampled image to obtain the third fused feature image; The first fused feature image, the second fused feature image, and the third fused feature image are respectively connected to the corresponding first detection head, second detection head, and third detection head, and the recognition result is obtained based on the output features of each detection head.

4. A monitoring and identification device based on machine vision, characterized in that, The device includes: Image acquisition module: Acquires an image of the target area, acquires all pixels of the image, and extracts the pixel values ​​corresponding to all pixels; Pixel segmentation module: Divides pixels into different states based on pixel values ​​and preset pixel thresholds. The different states of pixels include strong pixels, weak pixels, and normal pixels. Image enhancement module: dynamically adjusts the corresponding learning weight coefficients according to pixels in different states; performs image enhancement operations based on the learning weight coefficients to obtain the target image; Image recognition module: Substitutes the target image into the target recognition model to obtain the recognition result; the recognition result is whether the behavioral characteristics of the target object exist in the target area; based on the recognition result of the target image, determines the abnormal behavioral characteristics of the target object; The image enhancement module includes: a first weight module, a second weight module, and a learned weight partitioning module. The first weighting module is used to calculate the average value of the pixel values ​​corresponding to all strong pixels to obtain the average strong pixel value, and to substitute the average strong pixel value into the target training neural network model to obtain the first learning weight coefficient. The first weighting module is used to calculate the average weak pixel value by averaging the pixel values ​​corresponding to all weak pixels, and to substitute the average weak pixel value into the target training neural network model to obtain the second learning weight coefficient. The learning weight partitioning module includes learning weight coefficients, including a first learning weight coefficient and a second learning weight coefficient. The step of performing image enhancement operations based on the learned weight coefficients to obtain the target image includes: S1: The formula for calculating the enhancement operation is: Where LS(x) represents the pixel value output after image adjustment, Y(x) represents the normalized input pixel value, x represents the pixel point, and ρ represents the learning weight coefficient. This represents the pixel value output after the nth enhancement, where n represents the total number of enhancements. S2: Perform step S1 on all strong pixels and all weak pixels; when the number of iterations reaches n, output the enhanced image, which is denoted as the target image.

5. The monitoring and identification device based on machine vision according to claim 4, characterized in that, The pixel segmentation module includes: a threshold segmentation module, a first judgment module, a second judgment module, and a third judgment module. The threshold division module includes a preset pixel threshold, which comprises a lower limit threshold and an upper limit threshold for preset pixels. The first judgment module is used to determine if a pixel is a strong pixel if the pixel value is greater than or equal to the upper limit threshold. The second judgment module is used to determine if a pixel is a normal pixel if the lower threshold is less than the pixel value and the upper threshold is less than the upper threshold. The third judgment module is used to determine if a pixel is a weak pixel if its value is less than or equal to the lower threshold.

6. The monitoring and identification device based on machine vision according to claim 4, characterized in that, The image recognition module includes: a downsampling module, a first fusion module, a second fusion module, a third fusion module, and a detection head module. The downsampling module is used to take the target image as input to the target recognition model; and to perform continuous downsampling operations on the target image to obtain the corresponding first sampled image, second sampled image and third sampled image respectively. The first fusion module is used to perform a fusion operation on the third sampled image to obtain a first fused feature image; The second fusion module is used to perform an upsampling operation on the first fusion feature image and then perform a fusion operation with the second sampled image to obtain a second fusion feature image; The third fusion module is used to perform an upsampling operation on the second fusion feature image and then perform a fusion operation with the first sampled image to obtain a third fusion feature image. Detection head module: Connects the first fused feature image, the second fused feature image, and the third fused feature image to the corresponding first detection head, second detection head, and third detection head, respectively, and obtains the recognition result based on the output features of each detection head.

Images