A factory safety detection method and system based on security video and image recognition

Abstract

The application relates to the technical field of safety detection, in particular to a factory safety detection method and system based on security video and image recognition, which comprises the following steps: establishing a plurality of key detection areas in the factory monitoring coverage range; performing area-by-area target detection on the plurality of key detection areas based on security video and factory images to obtain a region detection target set corresponding to each key detection area; and performing feature fusion on the position information and appearance features of each detection target in the region detection target set corresponding to each key detection area to obtain a region fusion feature of each key detection area. The abnormal behavior quantity in the region behavior sequence and the target effective coverage area in the region fusion feature are cross-judged, so that the monitoring coverage insufficient area caused by shielding or layout dead angle can be accurately identified.

Description

Technical Field

[0001] This invention relates to the field of security detection technology, specifically to a factory security detection method and system based on security video and image recognition. Background Technology

[0002] Currently, traditional methods rely entirely on the fixed installation locations and linear field-of-view coverage of physical monitoring equipment, failing to utilize the frequency of abnormal events in a region's behavioral sequence and the actual coverage area of ​​the target for cross-linking and judgment. When equipment is temporarily obstructed or blind spots exist, traditional systems can only record the absence of image data in that area, but cannot proactively detect whether this "no data" state is caused by insufficient coverage. More importantly, traditional methods completely lack data on the security status within monitoring blind spots and lack predictive reasoning mechanisms. They cannot leverage temporal proximity and scene similarity to retrieve reference information from historical or surrounding data to estimate blind spot risks, resulting in a permanent information gap in the overall security assessment in the spatial dimension. The assessment results suffer from systemic biases due to information gaps that cannot be remedied.

[0003] Furthermore, traditional methods typically perform a one-time evaluation and output the final conclusion after acquiring various detection data and scores, without introducing secondary triggering logic based on risk level thresholds. When certain areas are identified as high-risk, traditional systems do not construct them into a set to be processed and drive the original multi-source data for re-integration and evaluation; instead, they directly adopt the initial calculation results. This open-loop architecture causes accidental false detections, local feature extraction errors, or misjudgments caused by temporary occlusion in a single detection to be fixed and amplified along the process. It is impossible to correct the risk weight allocation of each area through repeated iterations, and the evaluation results cannot be guaranteed to converge to below the true safety level. This results in weak adaptability under different security situations, and the stability and decision-making reference value of the output results are seriously insufficient. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: a factory security detection method based on security video and image recognition, comprising: Within the factory monitoring coverage area, multiple key detection areas are established. Based on security videos and factory images, target detection is performed on each key detection area to obtain the set of area detection targets corresponding to each key detection area. Based on the location information and appearance features of each detection target in the target set corresponding to each key detection area, feature fusion is performed to obtain the region fusion feature of each key detection area. Based on the region fusion feature of each key detection area, behavior trajectory analysis is performed to obtain the region behavior sequence of each key detection area. Based on the regional behavior sequence and regional fusion characteristics of each key detection area, identify the blind spots in multiple key detection areas that are not covered by existing monitoring, and establish a blind spot set consisting of at least one blind spot. For each blind spot in the blind spot set, images with similar time periods and scenes to the blind spot are obtained from security videos and factory images as reference data. The safety hazard identification model is combined with the regional behavior sequence corresponding to the blind spot to generate the predicted detection result of the blind spot, resulting in a set of blind spot detection results containing each predicted detection result. Based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, a security status assessment of the factory is conducted to obtain the factory's security assessment results. Based on multiple key detection areas, the safety assessment results are analyzed regionally to obtain the regional safety scores for the corresponding key detection areas.

[0005] Preferably, after obtaining the regional security score for the corresponding key detection area, the method further includes: Based on multiple key detection areas, the safety assessment results are analyzed regionally to obtain the regional safety scores for the corresponding key detection areas; The risk level of each area security score is determined, and the risk level result of each area security score is obtained. It is also determined whether the number of high-risk areas does not exceed the preset first quantity threshold. If so, generate security test results based on the security scores of each region and end the process; If not, then based on the regional security scores of high-risk areas, establish a set of areas to be processed consisting of high-risk areas; Based on security videos, factory images, blind spot detection results, regional fusion features, and a set of areas to be processed, the factory's security status is reassessed to obtain updated security assessment results. The process then returns to perform regional analysis of the updated security assessment results according to multiple key detection areas.

[0006] Preferably, within the factory monitoring coverage area, multiple key inspection areas are established, including: Multiple grid points are evenly set on the factory floor plan, and the directions formed by the multiple grid points and the factory center point are used as the coverage parameters of the multiple key detection areas respectively. For any critical detection area, the boundary parameters of the critical detection area are obtained based on the coverage radius of the critical detection area, the coordinates of the factory center point, and the coverage range parameters of the critical detection area.

[0007] Preferably, based on security videos and factory images, target detection is performed on a region-by-region basis in multiple key detection areas to obtain a set of region detection targets corresponding to each key detection area, including: Keyframes are extracted from security videos to obtain multiple key video frames. These key video frames and factory images are then assigned to multiple key detection regions to obtain a set of region images corresponding to each key detection region. Target recognition is performed on the set of region images corresponding to each key detection region to obtain the set of region recognition targets corresponding to each key detection region. Based on the confidence of each target in the target set corresponding to each key detection region, a target candidate set is obtained for each key detection region. Then, based on the overlap of the bounding boxes of each candidate target in the target candidate set corresponding to each key detection region, duplicates are removed to obtain the target detection set for each key detection region.

[0008] Preferably, feature fusion is performed based on the location information and appearance features of each detected target in the target set corresponding to each key detection region to obtain the region fusion feature of each key detection region, including: Based on the location information of each detection target in the set of regional detection targets corresponding to each key detection area, the spatial coordinate vector of each key detection area is obtained; Based on the appearance features of each detection target in the set of regional detection targets corresponding to each key detection region, the appearance description vector of each key detection region is obtained. Feature concatenation is performed on the spatial coordinate vector and appearance description vector of each key detection region to obtain the region fusion feature of each key detection region.

[0009] Preferably, behavioral trajectory analysis is performed based on the region fusion features of each key detection region to obtain the region behavioral sequence of each key detection region, including: Based on the temporal variation trend of the regional fusion features of each key detection area, the motion trajectory sequence of each detected target in each key detection area is obtained; By comparing the motion trajectory sequence of each key detection area with the preset safety trajectory template, the trajectory deviation value of each key detection area is obtained. When the trajectory deviation value exceeds the preset deviation threshold, the corresponding detection target in the key detection area is marked as abnormal behavior, and the regional behavior sequence of each key detection area is obtained based on the temporal arrangement of all abnormal behaviors.

[0010] Preferably, based on the regional behavior sequence and regional fusion characteristics of each key detection area, blind spots not covered by existing monitoring are identified in multiple key detection areas, and a blind spot set consisting of at least one blind spot is established, including: For any one of the multiple key detection areas, based on the number of abnormal behaviors in the regional behavior sequence of the key detection area and the effective coverage area of ​​each detection target in the regional fusion feature of the key detection area, it is determined whether the key detection area has insufficient monitoring coverage. When the number of abnormal behaviors in the key detection area exceeds the preset behavior threshold and the effective coverage area is lower than the preset area threshold, the key detection area is determined to be a blind spot not covered by existing monitoring. All key detection areas identified as blind spots are grouped into a blind spot set.

[0011] Preferably, for each blind spot in the blind spot set, images with similar time periods and scenes to the blind spot are obtained from security videos and factory images as reference data. A security hazard identification model is then used in conjunction with the corresponding regional behavior sequence to generate a predicted detection result for that blind spot, resulting in a blind spot detection result set containing all predicted detection results, including: For each blind spot in the blind spot set, the image with the highest scene similarity to the abnormal behavior in the regional behavior sequence of that blind spot is obtained from security videos and factory images and used as the reference data for that blind spot; Obtain the spatiotemporal correlation between the reference data of the blind zone and the regional behavior sequence of the blind zone; The spatiotemporal correlation and the reference data are input into the safety hazard identification model to obtain the predicted detection result of the blind spot generated by the safety hazard identification model; After obtaining the prediction detection results for each blind zone in the blind zone set, a blind zone detection result set containing each prediction detection result is obtained.

[0012] Preferably, based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, a security status assessment of the factory is conducted to obtain the factory's security assessment results, including: Based on the regional fusion features of multiple key detection areas, a regional security feature vector for each key detection area is obtained; Based on the risk level of each predicted detection result in the blind zone detection result set, the blind zone risk value of each blind zone is obtained; The regional security score for each key detection area is obtained by comprehensively calculating the regional security feature vector of each key detection area and the blind risk value of the corresponding blind area. The factory's safety assessment results are obtained by weighted fusion of regional safety scores from all key inspection areas.

[0013] A factory security inspection system based on security video and image recognition, applicable to the aforementioned factory security inspection method based on security video and image recognition, includes: The target detection unit is configured to establish multiple key detection areas within the factory monitoring coverage area, and perform area-by-area target detection based on security video and factory images to obtain the area detection target set corresponding to each key detection area. The trajectory analysis unit is configured to perform feature fusion based on the position information and appearance features of each detection target in the set of regional detection targets corresponding to each key detection area, to obtain the regional fusion features of each key detection area, and to perform behavioral trajectory analysis based on the regional fusion features of each key detection area to obtain the regional behavior sequence of each key detection area. The blind spot determination unit is configured to determine each blind spot in multiple key detection areas that is not covered by existing monitoring based on the regional behavior sequence and regional fusion features of each key detection area, and to establish a blind spot set consisting of at least one blind spot. The blind spot detection unit is configured to, for each blind spot in the blind spot set, obtain images from security videos and factory images that are close in time and similar in scene to the blind spot as reference data, and use a safety hazard identification model combined with the regional behavior sequence corresponding to the blind spot to generate a predictive detection result for the blind spot, thereby obtaining a set of blind spot detection results containing each predictive detection result; The status assessment unit is configured to perform a safety status assessment of the factory based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, and obtain the factory's safety assessment results. The regional analysis unit is configured to perform regional analysis on the safety assessment results according to multiple key detection areas, and obtain the regional safety score under the corresponding key detection areas.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention accurately identifies areas with insufficient monitoring coverage due to obstruction or blind spots by cross-judging the number of abnormal behaviors in a region's behavioral sequence with the effective coverage area of ​​the target in the region's fusion features. Based on this, for the identified blind spots, a dual retrieval mechanism of temporal proximity and scene visual similarity is used to obtain reference data. Furthermore, a spatiotemporal correlation is established by combining the behavioral sequences corresponding to the blind spots, and a safety hazard identification model is used to generate predictive detection results for the blind spots. This enables proactive reasoning about potential risk states within blind spots, effectively compensating for the field-of-view deficiencies of physical monitoring equipment, significantly improving the perceptual completeness and data integrity of safety assessments in the spatial dimension, and avoiding overall assessment bias caused by missing information. This invention introduces a comparison logic between risk level determination and a threshold for the number of high-risk areas after obtaining security scores for each region. When the number of high-risk areas exceeds a preset threshold, instead of directly outputting a one-time conclusion, the high-risk areas are constructed into a set to be processed, and multi-source data such as original security videos, factory images, blind spot detection results, and regional fusion features are re-evaluated to form a feedback iterative loop. This dynamic closed-loop mechanism enables the evaluation system to automatically strengthen the focus weight of key areas under high-risk conditions, and gradually correct the risk weight allocation of each region through repeated iterative calculations. This effectively suppresses the accumulation of errors and local misjudgments that may occur in a single static evaluation, and ultimately ensures that the evaluation results converge within the safety threshold, significantly enhancing the adaptability of the overall detection method under different security conditions and the stability of the result output. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture in one embodiment of the present invention.

[0016] In the diagram: 1. Target detection unit; 2. Trajectory analysis unit; 3. Blind spot determination unit; 4. Blind spot detection unit; 5. State assessment unit; 6. Region analysis unit. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1This invention provides a technical solution: a factory security detection method based on security video and image recognition, comprising: S1. Within the factory monitoring coverage area, establish multiple key detection areas. Based on security videos and factory images, perform area-by-area target detection on multiple key detection areas to obtain the set of area detection targets corresponding to each key detection area. S2. Based on the location information and appearance features of each detection target in the set of regional detection targets corresponding to each key detection area, feature fusion is performed to obtain the regional fusion features of each key detection area. Based on the regional fusion features of each key detection area, behavior trajectory analysis is performed to obtain the regional behavior sequence of each key detection area. S3. Based on the regional behavior sequence and regional fusion characteristics of each key detection area, identify the blind spots in multiple key detection areas that are not covered by existing monitoring, and establish a blind spot set consisting of at least one blind spot. S4. For each blind spot in the blind spot set, obtain images with similar time and scene to the blind spot from security videos and factory images as reference data, and use the security hazard identification model combined with the regional behavior sequence corresponding to the blind spot to generate the predicted detection result of the blind spot, and obtain a set of blind spot detection results containing each predicted detection result. S5. Based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, a security status assessment of the factory is conducted to obtain the factory's security assessment results. S6. Based on multiple key detection areas, perform regional analysis on the safety assessment results to obtain the regional safety score for each key detection area.

[0019] In an optional embodiment, after obtaining the regional security score under the corresponding key detection area, the method further includes: Based on multiple key detection areas, the safety assessment results are analyzed regionally to obtain the regional safety scores for the corresponding key detection areas; The risk level of each area security score is determined, and the risk level result of each area security score is obtained. It is also determined whether the number of high-risk areas does not exceed the preset first quantity threshold. If so, generate security test results based on the security scores of each region and end the process; If not, then based on the regional security scores of high-risk areas, establish a set of areas to be processed consisting of high-risk areas; Based on security videos, factory images, blind spot detection results, regional fusion features, and a set of areas to be processed, the factory's security status is reassessed to obtain updated security assessment results. The process then returns to perform regional analysis of the updated security assessment results according to multiple key detection areas.

[0020] It should be noted that, based on the preset risk level classification standard, the security score of each area is mapped to the corresponding risk level, which includes at least high-risk and non-high-risk levels. The number of areas judged as high-risk is counted and compared with a preset first threshold. If the number of high-risk areas is not greater than the first threshold, an overall security status detection report is generated directly based on the security scores of all areas, and the entire detection process ends. If the number of high-risk areas is greater than the first threshold, all high-risk areas are extracted to form a set of areas to be processed. Then, using the original video surveillance data, static scene image data, the obtained blind spot detection result set, the fusion feature data of each area, and the set of areas to be processed, the overall monitoring space is re-evaluated to obtain an updated security assessment result. The updated security assessment result is then regionalized according to all key monitoring areas to obtain a new security score for each area, and the process returns to the risk level determination step to repeat the above comparison and processing operations. This loop continues until the condition that the number of high-risk areas does not exceed the first threshold or the preset maximum number of iterations is reached, at which point the final security detection result is output.

[0021] In an optional embodiment, multiple key detection zones are established within the factory monitoring coverage area, including: Multiple grid points are evenly set on the factory floor plan, and the directions formed by the multiple grid points and the factory center point are used as the coverage parameters of the multiple key detection areas respectively. For any critical detection area, the boundary parameters of the critical detection area are obtained based on the coverage radius of the critical detection area, the coordinates of the factory center point, and the coverage range parameters of the critical detection area.

[0022] It should be noted that the process involves obtaining a planar layout of the target monitoring space, and setting multiple sampling points on this layout in a uniformly spaced manner. Then, using the geometric center point of the target monitoring space as the reference origin, the orientation of each sampling point relative to the reference origin is determined, and this orientation is used as the coverage orientation parameter of the corresponding key detection area. For any key detection area, based on the preset coverage distance radius, the spatial coordinates of the reference origin, and the coverage orientation parameter of the area itself, the boundary limit of the area on the plane is calculated using polar coordinate conversion or vector synthesis methods. This boundary limit includes the coordinates of the outer edge point or the equation of the boundary line segment of the area, thereby uniquely determining the spatial coverage range of the key detection area. By traversing all sampling points, the boundary parameters of all key detection areas are obtained, thus completing the establishment of the key detection areas.

[0023] In an optional embodiment, based on security video and factory images, target detection is performed on a region-by-region basis across multiple key detection areas to obtain a set of region detection targets corresponding to each key detection area, including: Keyframes are extracted from security videos to obtain multiple key video frames. These key video frames and factory images are then assigned to multiple key detection regions to obtain a set of region images corresponding to each key detection region. Target recognition is performed on the set of region images corresponding to each key detection region to obtain the set of region recognition targets corresponding to each key detection region. Based on the confidence of each target in the target set corresponding to each key detection region, a target candidate set is obtained for each key detection region. Then, based on the overlap of the bounding boxes of each candidate target in the target candidate set corresponding to each key detection region, duplicates are removed to obtain the target detection set for each key detection region.

[0024] It should be noted that continuously acquired surveillance video streams and static scene images are acquired, and the two together constitute the raw observation data; key frame images are extracted from the surveillance video stream according to time intervals or the degree of scene change to form a key frame sequence; the key frame sequence and static scene images are assigned to multiple pre-divided key monitoring areas, and the assignment is based on the overlap between the spatial range covered by the content of each image and the spatial position of each key monitoring area, thereby generating a corresponding regional image dataset for each key monitoring area; For each region's image dataset, a target recognition operation is performed. This operation uses a trained target detection model to output the category labels, spatial location bounding boxes, and confidence scores of all identifiable objects within each region, forming a preliminary set of identified targets for that region. The initial target set for each region is filtered by confidence level. Targets with confidence levels below a preset threshold are removed, and targets with confidence levels meeting the requirements are retained, resulting in a candidate target set for that region. Next, the overlap of the bounding boxes of any two targets in the candidate target set is calculated. If the overlap exceeds a preset deduplication threshold, the two targets are considered the same object, and only the target with the highest confidence level is retained, while other duplicates are removed. After traversing all candidate targets in the region, the final target set for the region is obtained. This set contains the spatial location bounding box, category attributes, and confidence level information for each detected target, and each target uniquely corresponds to one entity object within the region. By sequentially performing the above image allocation, target recognition, confidence filtering, and overlap deduplication operations on all key monitoring areas, the corresponding set of regional detection targets for each area can be obtained.

[0025] In an optional embodiment, feature fusion is performed based on the location information and appearance features of each detection target in the set of detection targets corresponding to each key detection region to obtain the region fusion feature of each key detection region, including: Based on the position information of each detection target in the set of regional detection targets corresponding to each key detection area, the spatial coordinate vector of each key detection area is obtained; Based on the appearance features of each detection target in the set of regional detection targets corresponding to each key detection region, the appearance description vector of each key detection region is obtained. Feature concatenation is performed on the spatial coordinate vector and appearance description vector of each key detection region to obtain the region fusion feature of each key detection region.

[0026] It should be noted that after obtaining the target set corresponding to each key detection region, feature fusion is performed independently for each region. For any key detection region, the position coordinates of all target objects in its target set in the monitoring space are first extracted. The horizontal and vertical coordinates of each target are read sequentially according to their arrangement in the set, and these coordinates are concatenated to form a high-dimensional numerical sequence, which is the spatial coordinate vector of the region. At the same time, for each target object in the same set, a pre-trained appearance feature extraction network is used to extract a fixed-length feature descriptor from the corresponding image sub-region. The appearance descriptors of all targets are arranged in the same target order. By concatenating or statistically aggregating values, an appearance description vector for the region is formed. Then, the spatial coordinate vector of the region is concatenated with the appearance description vector along the feature dimension to generate a new joint feature vector. This joint feature vector contains both spatial distribution information and appearance semantic information, serving as the region fusion feature for the key detection region. The above spatial coordinate vector extraction, appearance description vector extraction, and feature concatenation operations are repeated for all key detection regions to obtain the region fusion features for each region, which are used for subsequent behavior trajectory analysis and security status assessment. This fusion process does not rely on prior knowledge of a specific scene; it only uses the original observation data of the detected targets within the region to perform a unified feature representation.

[0027] In an optional embodiment, behavioral trajectory analysis is performed based on the region fusion features of each key detection region to obtain a region behavior sequence for each key detection region, including: Based on the temporal variation trend of the regional fusion features of each key detection area, the motion trajectory sequence of each detected target in each key detection area is obtained; By comparing the motion trajectory sequence of each key detection area with the preset safety trajectory template, the trajectory deviation value of each key detection area is obtained. When the trajectory deviation value exceeds the preset deviation threshold, the corresponding detection target in the key detection area is marked as abnormal behavior, and the regional behavior sequence of each key detection area is obtained based on the temporal arrangement of all abnormal behaviors.

[0028] It should be noted that after obtaining the region fusion features of each key detection area, the numerical changes of these features over consecutive time frames are analyzed. The spatial coordinate components in the region fusion features form a sequence of movement trajectory points of each detected target within the monitoring space as time progresses. These trajectory points are sorted by timestamp and connected to obtain the motion trajectory sequence of each detected target. Simultaneously, several standard motion trajectory templates are pre-stored, corresponding to typical movement paths under normal conditions. The actual motion trajectory sequence of each detected target is compared with the most matching standard trajectory template within its region using point-by-point distance calculation or shape similarity measurement to obtain the trajectory deviation value of that target. This deviation value is then compared with the pre-defined... A deviation threshold is set for comparison. If the deviation value exceeds the threshold, the target's current behavior is determined to deviate from the normal pattern, and it is marked as abnormal behavior. After completing the trajectory comparison and marking of all detected targets in the area, all targets marked as abnormal behavior are arranged in chronological order of their occurrence to form the regional behavior sequence of the key detection area during the observation period. This behavior sequence records the temporal relationship of the abnormal behavior, the target identity, and the degree of deviation, serving as the basis for subsequent blind zone inference and risk assessment. By independently performing the above trajectory extraction, template comparison, deviation judgment, and abnormal mark arrangement operations on each key detection area, the regional behavior sequence of each area can be obtained.

[0029] In an optional embodiment, based on the regional behavior sequence and regional fusion characteristics of each key detection area, blind spots not covered by existing monitoring are identified in multiple key detection areas, and a blind spot set consisting of at least one blind spot is established, including: For any one of the multiple key detection areas, based on the number of abnormal behaviors in the regional behavior sequence of the key detection area and the effective coverage area of ​​each detection target in the regional fusion feature of the key detection area, it is determined whether the key detection area has insufficient monitoring coverage. When the number of abnormal behaviors in the key detection area exceeds the preset behavior threshold and the effective coverage area is lower than the preset area threshold, the key detection area is determined to be a blind spot not covered by existing monitoring. All key detection areas identified as blind spots are grouped into a blind spot set.

[0030] It should be noted that after obtaining the regional behavior sequences and regional fusion features of each key monitoring zone, a blind zone determination operation is performed independently for each zone. For any key monitoring zone, firstly, the total number of events carrying abnormal behavior markers in the regional behavior sequence of that zone is counted. Simultaneously, the spatial distribution range of all detected targets is extracted from the regional fusion features of that zone, and the effective coverage area actually occupied by these targets within the zone is calculated. This area is obtained by projecting the bounding boxes of each target into a union. Then, the total number of abnormal behavior events in that zone is compared with a preset behavior quantity threshold, and the effective coverage area is compared with a preset area lower limit threshold. If the total number of abnormal behavior events in that zone exceeds the behavior quantity threshold and the effective coverage area is lower than the area lower limit threshold, then the zone is determined to have insufficient sensor coverage and is identified as a spatial blind zone not directly covered by existing observation equipment. After traversing all key monitoring zones, all zones determined to be blind zones are aggregated to construct a blind zone set, which contains one or more blind zone units. This blind zone set is used for subsequent predictive detection and overall security assessment to ensure that the risks of unobserved areas can be estimated and addressed. Each blind zone unit retains the spatial location attributes and behavior characteristics of its respective zone. Information gathering is conducted as follows: After obtaining the regional behavior sequences and regional fusion features of each key monitoring zone, a blind zone determination operation is performed independently for each zone. For any key monitoring zone, the total number of events carrying abnormal behavior markers in the regional behavior sequence of that zone is first counted. At the same time, the spatial distribution range of all detected targets is extracted from the regional fusion features of that zone, and the effective coverage area actually occupied by these targets within the zone is calculated. This area is obtained by projecting the bounding boxes of each target into a union. Then, the total number of abnormal behavior events in that zone is compared with a preset behavior quantity threshold, and the effective coverage area is compared with a preset area lower limit threshold. If the total number of abnormal behavior events in that zone exceeds the behavior quantity threshold and the effective coverage area is lower than the area lower limit threshold, then it is determined that there is insufficient sensor coverage in that zone, and it is identified as a spatial blind zone not directly covered by existing observation equipment. After traversing all key monitoring zones, all zones identified as blind zones are aggregated to construct a blind zone set, which contains one or more blind zone units. This blind zone set is used for subsequent predictive detection and overall security assessment to ensure that the risks of unobserved areas can be estimated and handled. Each blind zone unit retains the spatial location attributes and behavioral feature information of its respective zone.

[0031] In an optional embodiment, for each blind spot in the blind spot set, images with similar time periods and scenes to the blind spot are obtained from security videos and factory images as reference data. A security hazard identification model is then used in conjunction with the corresponding regional behavior sequence to generate a predicted detection result for the blind spot, resulting in a blind spot detection result set containing all predicted detection results, including: For each blind spot in the blind spot set, the image with the highest scene similarity to the abnormal behavior in the regional behavior sequence of that blind spot is obtained from security videos and factory images and used as the reference data for that blind spot; Obtain the spatiotemporal correlation between the reference data of the blind zone and the regional behavior sequence of the blind zone; The spatiotemporal correlation and the reference data are input into the safety hazard identification model to obtain the predicted detection result of the blind spot generated by the safety hazard identification model; The predicted detection results of each blind zone in the blind zone set are aggregated to obtain a blind zone detection result set containing each predicted detection result.

[0032] It should be noted that for each blind spot unit in the blind spot set, firstly, all events marked as abnormal behavior are extracted from the corresponding regional behavior sequence, and the timestamp and spatial location of each abnormal event are obtained. Then, in all the collected continuous monitoring video streams and static scene image data, a search is performed based on temporal proximity and scene visual similarity. Temporal proximity refers to the difference between the image acquisition time and the time of the abnormal behavior occurrence being less than a set time limit. Scene visual similarity is measured by comparing the overall illumination intensity, background texture distribution, fixed obstacle outlines, and color statistical features of the images. From the search results, the frame or multiple consecutive frames that best match the time and spatial location of the abnormal behavior occurrence and have the highest similarity score are selected as reference data for that blind spot unit. Next, a spatiotemporal correlation is established between this reference data and the regional behavior sequence, which includes... The degree of overlap between the spatial coordinates of identifiable objects in the reference data and the historical trajectories of abnormal targets in the sequence, the consistency of the event sequence on the time axis, and the degree of coincidence of the movement direction form a correlation vector describing the mutual mapping between the two. This correlation vector and the reference data are input into a pre-constructed safety hazard identification model. This model adopts an encoder-decoder structure to fuse and infer the multi-source information input, and outputs the possible object categories, probability heatmaps of object occurrence locations, and predicted labels of abnormal behavior types that may exist in the blind zone unit. These outputs together constitute the prediction detection results of the blind zone. Each blind zone unit in the blind zone set is processed sequentially according to the above process, and the prediction detection results output by each unit are merged according to the blind zone identifier. Finally, a blind zone detection result set containing all blind zone prediction information is formed. This set provides supplementary risk data for unobserved areas for subsequent overall safety assessment.

[0033] In an optional embodiment, a security status assessment of the factory is performed based on security video, factory images, blind spot detection result sets, and the region fusion features of multiple key detection areas, to obtain the factory's security assessment results, including: Based on the regional fusion features of multiple key detection areas, a regional security feature vector for each key detection area is obtained; Based on the risk level of each predicted detection result in the blind zone detection result set, the blind zone risk value of each blind zone is obtained; The regional security score for each key detection area is obtained by comprehensively calculating the regional security feature vector of each key detection area and the blind risk value of the corresponding blind area. The factory's safety assessment results are obtained by weighted fusion of regional safety scores from all key inspection areas.

[0034] It should be noted that after obtaining the regional fusion features of all key monitoring areas and the blind spot detection result set, an overall security status assessment is performed. First, for each key monitoring area, multi-dimensional statistics reflecting the security situation are extracted from its regional fusion features, including the instantaneous density of detected targets within the area, the consistency index of movement direction, the degree to which the target dwell time deviates from the mean, and the frequency of targets entering and leaving the area. These statistics are combined according to a preset dimensional order to form the regional security feature vector for that area. Simultaneously, risk quantification is performed on each blind spot prediction result in the blind spot detection result set. Based on the object type risk coefficient, the severity level of the abnormal behavior prediction label, and the peak intensity in the location probability heatmap included in the prediction results, and referring to a pre-established risk level mapping table, each blind spot is assigned a numerical blind spot risk value. Then, for each key monitoring area, if the area is in the blind spot set... If a corresponding blind zone unit exists, the regional security feature vector of that area is comprehensively calculated together with the blind zone risk value of the corresponding blind zone. The calculation method is to multiply each component of the regional security feature vector by the normalization coefficient of the blind zone risk value and then sum them up, or to add the blind zone risk value as an independent term to the weighted sum of the regional security feature vectors to obtain the regional security score of that area. If no corresponding blind zone unit exists in that area, the regional security score is directly calculated based on the weighted modulus of its regional security feature vector. Finally, according to the area ratio of each key monitoring area in the overall monitoring space or the preset importance coefficient, a fusion weight is assigned to the regional security score of each area, and a weighted summation or weighted average operation is performed to obtain the overall security assessment result of the monitoring space. This security assessment result includes a global comprehensive risk index and security sub-item values ​​subdivided by risk category, which are used to characterize the overall security status level of the current monitoring space.

[0035] Example 2, please refer to Figure 2 This invention provides a technical solution: a factory security detection system based on security video and image recognition, applicable to the aforementioned factory security detection method based on security video and image recognition, comprising: Target detection unit 1 is configured to establish multiple key detection areas within the factory monitoring coverage area, and perform area-by-area target detection on multiple key detection areas based on security video and factory images to obtain the area detection target set corresponding to each key detection area; The trajectory analysis unit 2 is configured to perform feature fusion based on the position information and appearance features of each detection target in the set of regional detection targets corresponding to each key detection area to obtain the regional fusion features of each key detection area, and perform behavioral trajectory analysis based on the regional fusion features of each key detection area to obtain the regional behavior sequence of each key detection area. Blind spot determination unit 3 is configured to determine each blind spot in multiple key detection areas that is not covered by existing monitoring based on the regional behavior sequence and regional fusion features of each key detection area, and establish a blind spot set consisting of at least one blind spot. Blind spot detection unit 4 is configured to, for each blind spot in the blind spot set, obtain images from security videos and factory images that are close in time and similar in scene to the blind spot as reference data, and use the safety hazard identification model combined with the regional behavior sequence corresponding to the blind spot to generate the predicted detection result of the blind spot, thereby obtaining a set of blind spot detection results containing each predicted detection result; The status assessment unit 5 is configured to perform a safety status assessment of the factory based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, and obtain the factory's safety assessment results. Regional analysis unit 6 is configured to perform regional analysis on the safety assessment results according to multiple key detection areas, and obtain the regional safety score under the corresponding key detection areas.

[0036] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A factory security inspection method based on security video and image recognition, characterized in that, include: Within the factory monitoring coverage area, multiple key detection areas are established. Based on security videos and factory images, target detection is performed on each key detection area to obtain the set of area detection targets corresponding to each key detection area. Based on the location information and appearance features of each detection target in the target set corresponding to each key detection area, feature fusion is performed to obtain the region fusion feature of each key detection area. Based on the region fusion feature of each key detection area, behavior trajectory analysis is performed to obtain the region behavior sequence of each key detection area. Based on the regional behavior sequence and regional fusion characteristics of each key detection area, identify the blind spots in multiple key detection areas that are not covered by existing monitoring, and establish a blind spot set consisting of at least one blind spot. For each blind spot in the blind spot set, images with similar time periods and scenes to the blind spot are obtained from security videos and factory images as reference data. The safety hazard identification model is combined with the regional behavior sequence corresponding to the blind spot to generate the predicted detection result of the blind spot, resulting in a set of blind spot detection results containing each predicted detection result. Based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, a security status assessment of the factory is conducted to obtain the factory's security assessment results. Based on multiple key detection areas, the safety assessment results are analyzed regionally to obtain the regional safety scores for the corresponding key detection areas.

2. The factory security detection method based on security video and image recognition according to claim 1, characterized in that, After obtaining the regional security score for the corresponding key detection area, the method further includes: Based on multiple key detection areas, the safety assessment results are analyzed regionally to obtain the regional safety scores for the corresponding key detection areas; The risk level of each area security score is determined, and the risk level result of each area security score is obtained. It is also determined whether the number of high-risk areas does not exceed the preset first quantity threshold. If so, generate security test results based on the security scores of each region and end the process; If not, then based on the regional security scores of high-risk areas, establish a set of areas to be processed consisting of high-risk areas; Based on security videos, factory images, blind spot detection results, regional fusion features, and a set of areas to be processed, the factory's security status is reassessed to obtain updated security assessment results. The process then returns to perform regional analysis of the updated security assessment results according to multiple key detection areas.

3. The factory security detection method based on security video and image recognition according to claim 2, characterized in that, Within the factory's monitoring coverage area, multiple key inspection areas were established, including: Multiple grid points are evenly set on the factory floor plan, and the directions formed by the multiple grid points and the factory center point are used as the coverage parameters of the multiple key detection areas respectively. For any critical detection area, the boundary parameters of the critical detection area are obtained based on the coverage radius of the critical detection area, the coordinates of the factory center point, and the coverage range parameters of the critical detection area.

4. The factory security detection method based on security video and image recognition according to claim 3, characterized in that, Based on security videos and factory images, target detection is performed region-by-region in multiple key detection areas to obtain the set of target objects corresponding to each key detection area, including: Keyframes are extracted from security videos to obtain multiple key video frames. These key video frames and factory images are then assigned to multiple key detection regions to obtain a set of region images corresponding to each key detection region. Target recognition is performed on the set of region images corresponding to each key detection region to obtain the set of region recognition targets corresponding to each key detection region. Based on the confidence of each target in the target set corresponding to each key detection region, a target candidate set is obtained for each key detection region. Then, based on the overlap of the bounding boxes of each candidate target in the target candidate set corresponding to each key detection region, duplicates are removed to obtain the target detection set for each key detection region.

5. A factory security detection method based on security video and image recognition according to claim 4, characterized in that, Feature fusion is performed based on the location information and appearance features of each detected target in the target set corresponding to each key detection area. The region fusion features of each key detection region are obtained, including: Based on the location information of each detection target in the set of regional detection targets corresponding to each key detection area, the spatial coordinate vector of each key detection area is obtained; Based on the appearance features of each detection target in the set of regional detection targets corresponding to each key detection region, the appearance description vector of each key detection region is obtained. Feature concatenation is performed on the spatial coordinate vector and appearance description vector of each key detection region to obtain the region fusion feature of each key detection region.

6. A factory security detection method based on security video and image recognition according to claim 5, characterized in that, Behavioral trajectory analysis is performed based on the regional fusion features of each key detection region to obtain the regional behavioral sequence of each key detection region, including: Based on the temporal variation trend of the regional fusion features of each key detection area, the motion trajectory sequence of each detected target in each key detection area is obtained; By comparing the motion trajectory sequence of each key detection area with the preset safety trajectory template, the trajectory deviation value of each key detection area is obtained. When the trajectory deviation value exceeds the preset deviation threshold, the corresponding detection target in the key detection area is marked as abnormal behavior, and the regional behavior sequence of each key detection area is obtained based on the temporal arrangement of all abnormal behaviors.

7. A factory security detection method based on security video and image recognition according to claim 6, characterized in that, Based on the regional behavior sequence and regional fusion characteristics of each key detection area, blind spots not covered by existing monitoring are identified in multiple key detection areas, and a set of blind spots consisting of at least one blind spot is established, including: For any one of the multiple key detection areas, based on the number of abnormal behaviors in the regional behavior sequence of the key detection area and the effective coverage area of ​​each detection target in the regional fusion feature of the key detection area, it is determined whether the key detection area has insufficient monitoring coverage. When the number of abnormal behaviors in the key detection area exceeds the preset behavior threshold and the effective coverage area is lower than the preset area threshold, the key detection area is determined to be a blind spot not covered by existing monitoring. All key detection areas identified as blind spots are grouped into a blind spot set.

8. A factory security detection method based on security video and image recognition according to claim 7, characterized in that, For each blind spot in the blind spot set, images with similar times and scenes to the blind spot are obtained from security videos and factory images as reference data. A security hazard identification model is then used in conjunction with the corresponding regional behavior sequence to generate a predicted detection result for that blind spot, resulting in a blind spot detection result set containing all the predicted detection results, including: For each blind spot in the blind spot set, the image with the highest scene similarity to the abnormal behavior in the regional behavior sequence of that blind spot is obtained from security videos and factory images and used as the reference data for that blind spot; Obtain the spatiotemporal correlation between the reference data of the blind zone and the regional behavior sequence of the blind zone; The spatiotemporal correlation and the reference data are input into the safety hazard identification model to obtain the predicted detection result of the blind spot generated by the safety hazard identification model; The predicted detection results of each blind zone in the blind zone set are aggregated to obtain a blind zone detection result set containing each predicted detection result.

9. A factory security detection method based on security video and image recognition according to claim 8, characterized in that, Based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, a security status assessment of the factory is conducted, yielding the factory's security assessment results, including: Based on the regional fusion features of multiple key detection areas, a regional security feature vector for each key detection area is obtained; Based on the risk level of each predicted detection result in the blind zone detection result set, the blind zone risk value of each blind zone is obtained; The regional security score for each key detection area is obtained by comprehensively calculating the regional security feature vector of each key detection area and the blind risk value of the corresponding blind area. The factory's safety assessment results are obtained by weighted fusion of regional safety scores from all key inspection areas.

10. A factory security inspection system based on security video and image recognition, applicable to the factory security inspection method based on security video and image recognition as described in any one of claims 1-9, characterized in that, include: The target detection unit is configured to establish multiple key detection areas within the factory monitoring coverage area, and perform area-by-area target detection based on security video and factory images to obtain the area detection target set corresponding to each key detection area. The trajectory analysis unit is configured to perform feature fusion based on the position information and appearance features of each detection target in the set of regional detection targets corresponding to each key detection area, to obtain the regional fusion features of each key detection area, and to perform behavioral trajectory analysis based on the regional fusion features of each key detection area to obtain the regional behavior sequence of each key detection area. The blind spot determination unit is configured to determine each blind spot in multiple key detection areas that is not covered by existing monitoring based on the regional behavior sequence and regional fusion features of each key detection area, and to establish a blind spot set consisting of at least one blind spot. The blind spot detection unit is configured to, for each blind spot in the blind spot set, obtain images from security videos and factory images that are close in time and similar in scene to the blind spot as reference data, and use a safety hazard identification model combined with the regional behavior sequence corresponding to the blind spot to generate a predictive detection result for the blind spot, thereby obtaining a set of blind spot detection results containing each predictive detection result; The status assessment unit is configured to perform a safety status assessment of the factory based on security video, factory images, blind spot detection results, and the regional fusion features of multiple key detection areas, and obtain the factory's safety assessment results. The regional analysis unit is configured to perform regional analysis on the safety assessment results according to multiple key detection areas, and obtain the regional safety score under the corresponding key detection areas.

Images