Redundancy design method and system for 3D and AI visual sensing visible light movement

Abstract

The invention provides a redundancy design method and system for a 3D and AI visual sensing visible light movement, and the method and system are applied to a processing unit of the AI visual sensing visible light movement, and the method comprises the steps: obtaining first image data according to a preset first shooting parameter; based on a preset event analysis model, detecting whether an early warning event occurs in the first image data; if the early warning event occurs, detecting an upgrade probability that the early warning event is upgraded to a dangerous event according to a preset LSTM model, the LSTM model being a model trained based on a preset historical early warning event and a preset historical dangerous event; if the upgrading probability is greater than a preset upgrading threshold value, determining and executing an alarm prevention measure according to the dangerous event and a preset prevention measure set, and obtaining second image data according to a preset second shooting parameter; based on a preset event analysis model, detecting an occurrence probability of a dangerous event according to the second image data; and if the occurrence probability is lower than a preset probability threshold value, stopping executing the alarm prevention measures.

Description

Technical Field

[0001] This application relates to the field of visual sensing technology, and particularly to a redundancy design method and system for a 3D and AI visual sensing visible light module. Background Art

[0002] Currently, with the rapid development of artificial intelligence and machine vision technologies, the demand for AI visual sensors in various application scenarios is increasing day by day. The AI visual sensor integrates an operation module of an artificial intelligence algorithm and can directly output the results of machine vision algorithms. This method is called "In-Sensor Computing". This technology can greatly reduce the latency of data transmission and improve processing efficiency, and is widely used in fields such as autonomous driving, security monitoring, and industrial automation.

[0003] In many practical applications, AI visual sensors need to cope with dynamic and complex environments. Especially when a warning event occurs, they need to respond quickly and make correct judgments. For example, in the autonomous driving scenario, the sensor needs to detect obstacles and pedestrians on the road in time to prevent traffic accidents. However, when encountering difficult events, the sensor not only needs to frequently increase the image acquisition and computing power analysis frequencies, but may also face the problem of untimely discovery of warning events. This is mainly because traditional visual sensors rely on image acquisition and processing at a fixed frequency. When the event occurrence frequency exceeds the processing capacity of the system, it may lead to missed detection or delay of warning events. Summary of the Invention

[0004] This application provides a redundancy design method and system for a 3D and AI visual sensing visible light module, which is used to improve the detection ability of the AI visual sensing visible light module for warning events and improve the detection speed.

[0005] In a first aspect, an embodiment of this application provides a redundancy design method for a 3D and AI visual sensing visible light module. The method includes: Obtain first image data according to a preset first shooting parameter; Based on a preset event analysis model, detect whether a warning event appears in the first image data; If the warning event appears, detect the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events; If the escalation probability is greater than a preset escalation threshold, determine and execute a warning prevention measure according to the dangerous event and a preset set of prevention measures, and obtain second image data according to a preset second shooting parameter; Based on a preset event analysis model, detect the occurrence probability of the dangerous event according to the second image data; If the occurrence probability is lower than a preset probability threshold, stop executing the police situation prevention measure.

[0006] In a second aspect, an embodiment of the present application provides a redundant design system for a 3D and AI vision sensing visible light module, and the device includes: An image acquisition module, configured to acquire first image data according to preset first shooting parameters; An event detection module, configured to detect whether a warning event appears in the first image data based on a preset event analysis model; A probability calculation module, configured to, if the warning event appears, detect the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events; An event prevention module, configured to, if the escalation probability is greater than a preset escalation threshold, determine and execute a police situation prevention measure according to the dangerous event and a preset set of prevention measures, and acquire second image data according to preset second shooting parameters; A danger analysis module, configured to detect the occurrence probability of the dangerous event according to the second image data based on a preset event analysis model; A cancellation determination module, configured to stop executing the police situation prevention measure if the occurrence probability is lower than a preset probability threshold.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, and the electronic device includes a memory and a processor; The memory is used to store a computer program; The processor is configured to execute the computer program and, when executing the computer program, implement the redundant design method for the 3D and AI vision sensing visible light module according to any one of the embodiments of the present application.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to implement the redundant design method for the 3D and AI vision sensing visible light module according to any one of the embodiments of the present application.

[0009] The embodiment of the present application provides a redundancy design method for a 3D and AI vision sensing visible light module, which is applied to the processing unit of the AI vision sensing visible light module. The method includes: obtaining first image data according to a preset first shooting parameter; detecting whether a warning event appears in the first image data based on a preset event analysis model; if a warning event appears, detecting the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on a preset historical warning event and a preset historical dangerous event; if the escalation probability is greater than a preset escalation threshold, determining and executing a police situation prevention measure according to the dangerous event and a preset set of prevention measures, and obtaining second image data according to a preset second shooting parameter; detecting the occurrence probability of the dangerous event based on the preset event analysis model according to the second image data; if the occurrence probability is lower than a preset probability threshold, stopping the execution of the police situation prevention measure. Through the above method, when a warning event is detected by the event model, the warning event is input into the LSTM model for matching detection to confirm whether the event is a dangerous event. Compared with re-analyzing the dangerous event through an image after increasing the frame rate, the speed is faster. During the event analysis process, re-analyzing the dangerous event through an image after increasing the frame rate is still retained for two-way redundant comparison design to ensure the accuracy of the event analysis and improve the overall safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a schematic flowchart of a redundancy design method for a 3D and AI vision sensing visible light module provided by an embodiment of the present application; Figure 2 It is a schematic block diagram of a redundancy design system for a 3D and AI vision sensing visible light module provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0013] The flowcharts shown in the accompanying drawings are merely illustrative examples, not necessarily including all content and operations / steps, nor necessarily executed in the described order. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0014] It should also be understood that the terms used in the specification of this application are merely for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification of this application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0015] It should be further understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0016] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a redundancy design method for a 3D and AI vision sensing visible light module provided by an embodiment of this application. As Figure 1 shown, the specific steps of this method include: S101 - S106.

[0017] S101. Obtain first image data according to preset first shooting parameters.

[0018] Exemplarily, configure the parameters of the image sensor of the first visible light module, and perform pre - processing on the image data through multi - dimensional sensor parameter collaborative control, including exposure time, gain coefficient, frame rate, pixel reading mode, etc., and combine the dynamic range enhancement (DRE) algorithm and the image signal processor (ISP). Adopt multi - spectrum and multi - channel image fusion technology to enhance the acquired original image data in the spatial and frequency domains, extract multi - scale feature information such as the texture, gradient, and texture direction of the image, and construct a first image data set with high signal - to - noise ratio and high dynamic range. During the data acquisition process, ensure the stability and consistency of the image data through algorithms such as pixel - level adaptive denoising, color correction, and spectral equalization.

[0019] S102. Detect whether a warning event appears in the first image data based on a preset event analysis model.

[0020] Exemplarily, a multi-layer feature extraction model based on a deep convolutional neural network (DCNN) and an attention mechanism is constructed to perform multi-scale, multi-angle, and multi-dimensional feature mapping and semantic segmentation on the first image data. An improved object detection algorithm is adopted, combined with a spatio-temporal attention mechanism and a cross-scale feature fusion strategy, to extract potential abnormal regions and suspicious targets from the image data. A probability graph model based on Bayesian inference is introduced. By modeling the probability distribution of the image feature space, the conditional probability of each image region becoming a warning event is calculated, and a multi-dimensional and high-precision warning event recognition model is constructed in combination with the historical event library and the knowledge graph.

[0021] S103. If a warning event occurs, detect the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events.

[0022] Exemplarily, design a long short-term memory network based on a multi-layer temporal attention mechanism (Multi-layerAttentionLSTM) to construct a complex network structure including time dependence, context relevance, and event evolution rules. Through deep learning modeling of the temporal features, semantic features, and spatial features of historical warning events and dangerous events, multi-dimensional feature vectors are constructed. An improved attention weight calculation method is adopted, introducing a dynamic gating mechanism and an adaptive adjustment strategy to accurately capture the key turning points and potential risk factors of event evolution. Perform multi-scale and multi-modal probability inference on the escalation probability of the warning event, and output the escalation probability of the dangerous event based on the Bayesian probability framework.

[0023] S104. If the escalation probability is greater than a preset escalation threshold, determine and execute the warning prevention measures according to the dangerous event and a preset set of prevention measures, and obtain second image data according to the preset second shooting parameters.

[0024] Exemplarily, construct an intelligent decision-making system for dangerous event prevention measures based on a knowledge graph and a rule engine. By performing semantic association and dynamic weight assignment on a preset set of prevention measures, a set of prevention strategies with context adaptability is formed. Adopt a multi-agent collaborative decision-making algorithm to dynamically generate an optimal combination of prevention measures according to the characteristics, severity, and evolution trend of the dangerous event. While implementing the prevention measures, adjust the shooting parameters of the second visible light camera module, including adjusting the aperture, shutter speed, ISO sensitivity, etc., and preprocess the second image data using an image enhancement algorithm to improve the information capture ability and detail restoration degree of the image.

[0025] Since the ability of the AI vision sensing visible light module to judge events is usually carried out through the pixel brightness, edge structure of pixels, etc. in the image data, through the above steps, it is possible to prevent the AI vision sensing visible light module from frequently entering the warning mode, that is, when a warning event is detected, immediately analyze the dangerous event through the second image data, which can reduce the power consumption of the chip of the AI vision sensing visible light module.

[0026] S105. Based on the preset event analysis model, detect the occurrence probability of dangerous events according to the second image data.

[0027] Exemplarily, continue the deep convolutional neural network model constructed in S102, perform multi-scale feature extraction and semantic analysis on the second image data. Introduce a cross-modal feature fusion strategy based on the Graph Attention Network (GAT), and comprehensively analyze the spatio-temporal evolution features of the first and second image data. By constructing a dynamically evolving probability graph model, combining Bayesian inference and Markov Chain Monte Carlo (MCMC) methods, perform probability density estimation and dynamic update on the occurrence probability of dangerous events.

[0028] It should be noted that if the number and types of warning events detected by the preset LSTM model within a unit time, such as 10 minutes, exceed the preset quantity value, it indicates that the current environment may be a dangerous environment. Even if no dangerous event is detected, the preset event analysis model can be started in advance to continuously detect dangerous events. Through the above redundant design, the detection speed of dangerous events is improved, thereby enhancing safety.

[0029] S106. If the occurrence probability is lower than the preset probability threshold, stop executing the police situation prevention measures.

[0030] Exemplarily, design a probability judgment mechanism with adaptive threshold adjustment ability, combine dynamic confidence evaluation and risk sensitivity analysis, and make an intelligent decision on whether to terminate the prevention measures according to the confidence interval and risk propagation characteristics of the occurrence probability of dangerous events. Adopt fuzzy control theory and dynamic threshold learning algorithm to construct an intelligent judgment system that can autonomously adjust the preset according to real-time environmental changes, ensuring the accuracy and resource efficiency of the prevention measures.

[0031] The embodiment of the present application provides a redundancy design method for a 3D and AI vision sensing visible light module, which is applied to the processing unit of the AI vision sensing visible light module. The method includes: obtaining first image data according to a preset first shooting parameter; detecting whether a warning event appears in the first image data based on a preset event analysis model; if a warning event appears, detecting the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events; if the escalation probability is greater than a preset escalation threshold, determining and executing a police situation prevention measure according to the dangerous event and a preset set of prevention measures, and obtaining second image data according to a preset second shooting parameter; detecting the occurrence probability of the dangerous event based on the preset event analysis model according to the second image data; if the occurrence probability is lower than a preset probability threshold, stopping the execution of the police situation prevention measure. Through the above method, when a warning event is detected by the event model, the warning event is input into the LSTM model for matching detection to confirm whether the event is a dangerous event. Compared with re-analyzing the dangerous event through an image after increasing the frame rate, the speed is faster. During the event analysis process, re-analyzing the dangerous event through an image after increasing the frame rate is still retained for two-way redundant comparison design to ensure the accuracy of the event analysis and improve the overall safety.

[0032] To more clearly introduce the technical solution of the present application, the technical solution of the present application will also be introduced through specific embodiments below. It should be noted that the specific embodiment is used to expand the description of the technical solution of the present application, rather than limiting the present application.

[0033] In some embodiments, after detecting whether a warning event appears in the first image data based on the preset event analysis model, the method further includes: S107 - S112.

[0034] S107: Detect the similarity coefficient between the warning event and a preset historical event.

[0035] Exemplarily, this step can be implemented through a preset LSTM model.

[0036] S108: If the similarity coefficient is less than a preset similarity threshold, set the warning event as an unknown warning event, and obtain second image data according to a preset second shooting parameter.

[0037] Exemplarily, if an unknown warning event appears, it means that it is out of the processing range of the preset LSTM, and it is necessary to call the preset event analysis model for data processing.

[0038] S109: Detect whether an unknown dangerous event appears based on the preset event analysis model according to the second image data.

[0039] Exemplarily, before invoking the preset event analysis model, it is necessary to capture an image using the second shooting parameters to obtain second image data. Analyzing dangerous events with more and more detailed second image data helps improve the analysis accuracy of the model, thereby enhancing the prediction ability of dangerous events.

[0040] S110. If an unknown dangerous event is detected, determine and execute a super-standard preventive measure based on the unknown dangerous event and the preset set of preventive measures.

[0041] Exemplarily, in the production workshop of an intelligent factory, the robot control system continuously monitors the production environment through the visible light core of AI vision sensing. Suddenly, the sensor captures abnormal vibrations and temperature fluctuations, which are warning signals of unknown dangerous events. The system immediately activates the prevention mechanism, exceeding the normal operation process.

[0042] The robot control system quickly cross-analyzes the vibration frequency, temperature changes, and equipment operation data to determine that there may be a potential risk of equipment failure. According to the preset set of preventive measures, the system quickly takes super-standard measures: automatically reducing the production line speed, isolating the suspicious equipment area, and triggering the emergency ventilation and cooling systems.

[0043] At the same time, the robot control system sends a detailed warning report to the technical team, requesting an immediate on-site inspection. By taking preventive measures in advance, not only can potential production accidents be avoided, but also equipment losses and personnel safety risks can be minimized, demonstrating the accuracy and foresight of the intelligent warning system.

[0044] S111. Continuously detect unknown dangerous events based on the preset event analysis model until the unknown dangerous event degrades to a safe event.

[0045] S112. Store the unknown warning event and the unknown dangerous event in the training database of the preset LSTM model.

[0046] In some embodiments, the visible light core of AI vision sensing further includes: a first visible light core and a second visible light core. According to the preset first shooting parameters, obtain first image data, including: controlling the first visible light core to obtain left-view imaging data according to the preset first shooting parameters; controlling the second visible light core to obtain right-view imaging data according to the preset second shooting parameters; generating first image data based on the left-view imaging data and the right-view imaging data.

[0047] In some embodiments, based on a pre-set event analysis model, detecting whether a warning event appears in the first image data includes: performing a blurred line segment detection process on pixel points according to the first image data to obtain an initial blurred line segment set, where the initial blurred line segment set includes multiple blurred line segments; performing a time function fitting process on each blurred line segment according to the initial blurred line segment set to obtain a time function parameter set; performing a time averaging process on a pre-set motion blurred image according to the time function parameter set to obtain an average image; determining a representation tensor according to the average image and a pre-set event stream aligned with time; performing shallow feature extraction and fusion according to the representation tensor and the average image to obtain a preliminary fusion feature; performing a multi-scale encoder-decoder network process according to the preliminary fusion feature to obtain a target feature map, and the target feature map is used to calculate whether a warning event exists.

[0048] Exemplarily, first, preprocess the input first image data, including image normalization, noise reduction, etc. Use edge detection operators (such as Canny, Sobel, etc.) to extract image edges, and apply the Hough transform or other line segment detection algorithms to identify straight line segments in the image. Considering the characteristics of motion blur, perform a blur degree analysis on the detected line segments, and output an initial blurred line segment set containing information such as position, direction, and blur degree. Perform a time series analysis on each blurred line segment, establish a mathematical model (possibly using polynomial functions or trigonometric functions) describing the line segment motion trajectory, and use methods such as the least squares method for parameter fitting. Record time-related parameters such as motion speed and acceleration, and output a set containing time function parameters to describe motion characteristics. Based on the time function parameters, deduce the motion state at each moment. Before that, it is necessary to determine the motion blurred image, and the motion blurred image comes from the average of all clear images during the exposure duration. Perform a time domain integration on the motion blurred image, calculate the mean image in the time dimension, eliminate the influence of random noise and instantaneous disturbances, and obtain a stable average image representation. Align the average image with the pre-set event stream in time, construct a multi-dimensional tensor structure to store image and time series features. After alignment, perform feature mapping and encoding on the data, retain the relevance of spatio-temporal information, and output a unified representation tensor format. Use methods such as convolutional neural networks to extract the underlying features of the image, combine the time series information in the representation tensor, design a feature fusion strategy (such as an attention mechanism), integrate multi-modal information, and obtain a preliminarily fused feature representation. Construct a multi-level encoder network to extract deep features, restore the spatial detail information through a decoder network, perform feature extraction and reconstruction at different scales, add skip connections to retain detail information, and finally output a target feature map. Set a discrimination threshold based on the target feature map, use a classifier or detector for event recognition, calculate the probability score of the event occurrence, and judge whether to trigger a warning according to the pre-set rules, and output the judgment result of the warning event.

[0049] In some embodiments, based on the preliminary fusion features, a multi-scale encoder-decoder network is processed to obtain a target feature map, and the target feature map is used to calculate whether there is a warning event, including: 1) Based on the preliminary fusion features, multi-layer convolutional downsampling processing is performed on the fusion features to obtain a multi-scale feature set, and the multi-scale feature set includes a first-scale feature map, a second-scale feature map, and a third-scale feature map.

[0050] Specifically, based on the preliminary fusion feature F in , multi-layer convolutional downsampling processing is performed on the fusion features to obtain a multi-scale feature set {F s1 , F s2 , F s3}, and its calculation process is: F s1 = Pool(Conv(F in ; W1)); F s2 = Pool(Conv(F s1 ; W2)); F s3 = Pool(Conv(F s2 ; W3)); Among them, W1, W2, and W3 are convolutional kernel parameter matrices, W1 ∈ R {64×C×3×3} , W2 ∈ R {128×64×3×3} , W3 ∈ R {256 ×128×3×3} , the size is 3×3, Pool(·) represents the max pooling operation, the pooling kernel size is 2×2, and the stride is 2. F s1 , F s2 , F s3 are the feature maps of the 1 / 2, 1 / 4, and 1 / 8 downsampling scales respectively, and Conv(·) represents the convolutional operation, including the ReLU activation function.

[0051] 2) Based on the multi-scale feature set, skip connection transfer processing is performed on the third-scale feature map to obtain a high-level semantic feature set, and the high-level semantic feature set is associated with the first-scale feature map and the second-scale feature map through a weight sharing mechanism.

[0052] Specifically, based on the multi-scale feature set {F s1 , F s2 , F s3}, skip connection transfer processing is performed on the third-scale feature map F s3 to obtain a high-level semantic feature set H, and its calculation process is: H = α·ASPP(Fs3) + β·Skip(F s1 , F s2 ); Among them, ASPP(·) represents the atrous spatial pyramid pooling operation, with corresponding size of 3×3 and atrous rates of 1, 2, and 4 respectively. Skip(·) represents the skip connection operation. α and β are weight coefficients, adaptively calculated through the attention mechanism. ASPP(F s3 ) = Concat(Conv d1 (F s3 ), Conv d2 (F s3 ), Conv d4 (F s3 )) where Convdi represents the 3×3 atrous convolution with atrous rate i.

[0053] 3) According to the high-level semantic feature set and the second-scale feature map, perform cascaded decoding and upsampling processing on the feature map to obtain a sequence of intermediate feature maps, where the sequence of intermediate feature maps includes the first intermediate feature, the second intermediate feature, and the third intermediate feature.

[0054] Specifically, according to the high-level semantic feature set H and the second-scale feature map F s2 , perform cascaded decoding and upsampling processing on the feature map to obtain a sequence of intermediate feature maps {M1, M2, M3}, and its calculation process is as follows: M1 = Conv(Concat(Upsample(H), F s2 ); θ1) M2 = Conv(Upsample(M1); θ2) M3 = Conv(Concat(Upsample(M2), F s1 ); θ3); Among them, Upsample(·) represents the bilinear interpolation upsampling operation, which magnifies by 2 times; Concat(·) represents the feature map channel dimension concatenation operation; θ1, θ2, and θ3 are 1×1 convolution parameters; Conv(·) is the 3×3 convolution operation.

[0055] 4) According to the sequence of intermediate feature maps and the first-scale feature map, perform adaptive fusion processing on the features to obtain a multi-level fusion feature map, where the multi-level fusion feature map integrates spatial information of different scales through the residual connection structure.

[0056] Specifically, according to the sequence of intermediate feature maps {M1, M2, M3} and the first-scale feature map F s1 , perform adaptive fusion processing on the features to obtain a multi-level fusion feature map F fuse , and its calculation process is as follows: F use = SE(SPP(M3)) + ResBlock(F s1 ); Among them, SE(·) represents the Squeeze-and-Excitation channel attention module, SPP(·) represents the spatial pyramid pooling module, ResBlock(·) represents the residual connection module, SPP(M3)=Concat(AvgPool1(M3), AvgPool2(M3), AvgPool4(M3)), and AvgPooli represents the adaptive average pooling of size i×i.

[0057] 5) According to the multi-level fusion feature map, perform dense prediction processing on the fusion features to obtain the target feature map, and the target feature map is used to determine the location information and category information of the warning event after non-maximum suppression operation.

[0058] Specifically, according to the multi-level fusion feature map F fuse , perform dense prediction processing on the fusion features to obtain the target feature map F out , and its calculation process is as follows: F pred =Conv3×3(F fuse ; Wp) F prob =Softmax(Conv1×1(F pred ; Ws)) F out =NMS(Upsample(F prob )) ; Among them, Wp is the 3×3 convolution parameter matrix, Ws is the 1×1 convolution parameter matrix, Wp∈R {C×256×3×3} , Ws∈R {K ×C×1×1} , where K is the number of warning event categories. Softmax(·) represents the normalization function, NMS(·) represents the non-maximum suppression operation, and F out is the finally output target feature map.

[0059] In the above method, through the hierarchical processing structure of the multi-scale encoder-decoder network, the feature information of different levels is fully utilized. Among them, the skip connection mechanism reduces information loss, the adaptive fusion mechanism enhances feature expression, and the dense prediction structure provides fine-grained event detection ability, forming a complete technical link from feature extraction to event detection.

[0060] In some embodiments, according to the initial set of fuzzy line segments, perform time function fitting processing on each fuzzy line segment to obtain the set of time function parameters, including: S201 - S204.

[0061] S201. Discretize and resample the pixel coordinates of each fuzzy line segment according to the initial set of fuzzy line segments to obtain a set of resampled pixel points.

[0062] Exemplarily, by uniformly discretizing the pixel coordinates of the initial fuzzy line segment, the line segment is divided into a set of equally spaced sampling points along its length, ensuring that the coordinates of each fuzzy line segment have an equal sampling density. The resampling process uses quadratic interpolation to calculate the positions of the intermediate sampling points and generates a set of resampled pixel points with a uniform distribution.

[0063] S202. Perform linear regression fitting on the temporal changes of the pixel points according to the set of resampled pixel points to obtain an initial set of parameters of the fitted time function.

[0064] Exemplarily, based on the spatial coordinates of the discrete resampled points and the change in gray value associated with time, linear regression is performed on each fuzzy line segment using the least squares method to fit the initial parameters of the time function, including time offset, slope, and line segment direction. This step transforms the fitting error into a controllable set of parameters by constructing an error minimization equation.

[0065] S203. Perform quadratic optimization on the complex temporal characteristics of each fuzzy line segment according to the initial set of parameters of the fitted time function to obtain an optimized set of parameters of the time function.

[0066] Exemplarily, for the fitting result of the initial time function, a non - linear optimization algorithm (such as Newton's method or particle swarm algorithm) is applied to perform quadratic optimization on the parameters of the time function. The optimization objective is to minimize the projection error of the fuzzy line segment in the time domain, and finally generate an optimized set of parameters of the time function containing accurate parameters.

[0067] S204. Perform time parameter verification on each fuzzy line segment according to the optimized set of parameters of the time function to obtain a set of parameters of the time function.

[0068] Exemplarily, apply the optimized parameters of the time function to the original set of fuzzy line segments, and verify the parameter accuracy by calculating the spatial deviation and time error between the line segment and the fitted time function. Adopt a dynamic threshold adjustment strategy to eliminate the line segments with errors exceeding the limit, and output the final set of parameters of the time function after screening.

[0069] In some embodiments, according to the representation tensor and the average image, perform shallow feature extraction and fusion to obtain preliminary fusion features, including: S301 - S305.

[0070] S301. Separate the representation tensor to obtain a positive - polarity event tensor and a negative - polarity event tensor.

[0071] Exemplarily, the event spike tensor (EST) contains the time, position, and polarity information of the event stream. First, it is necessary to perform polarity separation processing on these event points, extract the positive-polarity event points and negative-polarity event points respectively, and form two separate tensors, namely the positive-polarity event tensor and the negative-polarity event tensor. The purpose of this step is to distinguish the event points with increasing brightness (positive polarity) and decreasing brightness (negative polarity) for subsequent processing.

[0072] S302. According to the positive-polarity event tensor and the negative-polarity event tensor, perform 7×7 convolution processing on each representation tensor respectively to obtain the positive-polarity preliminary feature map and the negative-polarity preliminary feature map.

[0073] Exemplarily, perform 7×7 convolution processing (the convolution kernel size is 7×7 and the stride is 2) on the separated positive-polarity event tensor and negative-polarity event tensor respectively. This will extract local spatial features. The convolution operation will generate two feature maps, namely the positive-polarity preliminary feature map and the negative-polarity preliminary feature map. Through the convolution operation, the local features of the event points can be obtained, which lays the foundation for subsequent feature fusion.

[0074] S303. According to the positive-polarity preliminary feature map and the negative-polarity preliminary feature map, perform connection processing in the channel dimension to obtain the merged feature map.

[0075] Exemplarily, perform connection processing on the positive-polarity preliminary feature map and the negative-polarity preliminary feature map in the channel dimension to form a merged feature map. The channel connection operation combines two separate feature maps into a multi-channel feature map, enabling the joint processing and analysis of positive and negative polarity features in the same feature map.

[0076] S304. According to the merged feature map and the average image, perform channel attention calculation processing to obtain the attention-weighted feature map.

[0077] Exemplarily, use the Channel Attention (CA) mechanism to process the merged feature map. The channel attention mechanism emphasizes important features and suppresses unimportant features by calculating the importance weights of each channel. The specific operation is to fuse the merged feature map with the average image, calculate the channel attention weights, and then apply these weights to the merged feature map to obtain the attention-weighted feature map.

[0078] S305. Input the attention-weighted feature map into a preset residual block, and through short connection and convolution operations, obtain the preliminary fusion feature.

[0079] Exemplarily, the attention-weighted feature map is input into a Residual Block for further feature refinement. Through skip connection and convolution operations, the Residual Block can effectively extract deep features and suppress the problem of feature degradation. After being processed by the Residual Block, preliminary fusion features are obtained, which will contain richer spatial and channel information and provide a basis for subsequent deep feature extraction and fusion.

[0080] In some embodiments, if a warning event occurs, the upgrade probability of the warning event being upgraded to a dangerous event is detected according to a preset LSTM model, including: extracting the features of the warning event according to the warning event to obtain a warning event feature vector; inputting the warning event feature vector into the LSTM model to obtain the output probability of the LSTM model; performing a threshold judgment process on the output probability according to the output probability of the LSTM model to obtain the upgrade probability; and performing an output process on the upgrade probability according to the upgrade probability to obtain the probability of the warning event being upgraded to a dangerous event.

[0081] Please refer to Figure 2 , Figure 2 FIG. is a schematic block diagram of a redundancy design system for a 3D and AI vision sensing visible light module. The redundancy design system 200 for the 3D and AI vision sensing visible light module is used to execute the foregoing redundancy design method for the 3D and AI vision sensing visible light module. Among them, the redundancy design system 200 for the 3D and AI vision sensing visible light module can be configured in a server.

[0082] Among them, the server can be an independent server, a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0083] As Figure 2 shown, the redundancy design system 200 for the 3D and AI vision sensing visible light module includes: an image acquisition module 201, an event detection module 202, a probability calculation module 203, an event prevention module 204, a risk analysis module 205, and a release determination module 206.

[0084] The image acquisition module 201 is configured to acquire first image data according to preset first shooting parameters.

[0085] The event detection module 202 is configured to detect whether a warning event appears in the first image data based on a preset event analysis model.

[0086] The probability calculation module 203 is used to detect the upgrade probability of the warning event to a dangerous event according to a preset LSTM model if the warning event occurs. The LSTM model is a model trained based on preset historical warning events and preset historical dangerous events.

[0087] The event prevention module 204 is used to determine and execute warning prevention measures according to the dangerous event and a preset set of prevention measures if the escalation probability is greater than a preset escalation threshold, and obtain second image data according to preset second shooting parameters.

[0088] The risk analysis module 205 is used to detect the occurrence probability of the dangerous event according to the second image data based on a preset event analysis model.

[0089] The release determination module 206 is used to stop executing the alarm prevention measures if the occurrence probability is lower than a preset probability threshold.

[0090] An embodiment of the present application provides an electronic device, which includes a memory and a processor; the memory is used to store computer programs; the processor is used to execute the computer program and implement a redundant design method for a 3D and AI visual sensing visible light movement such as any one of the embodiments of the present application when executing the computer program.

[0091] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements a redundant design method for a 3D and AI visual sensing visible light movement such as any one of the embodiments of the present application.

[0092] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A redundancy design method for a 3D and AI vision sensing visible light core unit, characterized in that A processing unit applied to an AI vision sensing visible light module, the method comprising: Obtaining first image data according to a preset first shooting parameter; Detecting whether a warning event appears in the first image data based on a preset event analysis model; If the warning event appears, detecting the escalation probability of the warning event being escalated to a dangerous event according to a preset LSTM model, where the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events; If the escalation probability is greater than a preset escalation threshold, determining and executing a police situation prevention measure according to the dangerous event and a preset set of prevention measures, and obtaining second image data according to a preset second shooting parameter; Detecting the occurrence probability of the dangerous event according to the second image data based on a preset event analysis model; If the occurrence probability is lower than a preset probability threshold, stop executing the police situation prevention measure.

2. The redundancy design method of the 3D and AI vision sensing visible light movement according to claim 1, characterized in that, After detecting whether a warning event appears in the first image data based on a preset event analysis model, the method further comprises: Detecting the similarity coefficient between the warning event and a preset historical event; If the similarity coefficient is less than a preset similarity threshold, setting the warning event as an unknown warning event, and obtaining second image data according to a preset second shooting parameter; Detecting whether an unknown dangerous event appears according to the second image data based on a preset event analysis model; If the unknown dangerous event is detected, determining and executing an over-specification prevention measure according to the unknown dangerous event and a preset set of prevention measures; Continuously detecting the unknown dangerous event based on the preset event analysis model until the unknown dangerous event is degraded to a safe event; Storing the unknown warning event and the unknown dangerous event into the training database of the preset LSTM model.

3. The redundant design method of the 3D and AI vision sensing visible light module according to claim 1, characterized in that, The AI vision sensing visible light module further comprises: a first visible light module and a second visible light module, and obtaining first image data according to a preset first shooting parameter comprises: Controlling the first visible light module to obtain left-view imaging data according to a preset first shooting parameter; Controlling the second visible light module to obtain right-view imaging data according to a preset second shooting parameter; Generating the first image data according to the left-view imaging data and the right-view imaging data.

4. The redundant design method of the 3D and AI vision sensing visible light module according to claim 3, wherein, Detecting whether a warning event appears in the first image data based on a preset event analysis model comprises: Performing a blurred line segment detection process on pixel points according to the first image data to obtain an initial blurred line segment set, where the initial blurred line segment set includes multiple blurred line segments; Performing a time function fitting process on each blurred line segment according to the initial blurred line segment set to obtain a time function parameter set; Performing a time averaging process on a preset motion blurred image according to the time function parameter set to obtain an average image; Determining a representation tensor according to the average image and a preset event stream aligned with time; Performing shallow feature extraction and fusion according to the representation tensor and the average image to obtain a preliminary fusion feature; Based on the preliminary fusion features, perform multi-scale encoder-decoder network processing to obtain a target feature map, which is used to calculate whether there is a warning event.

5. The redundancy design method of the 3D and AI vision sensing visible light module according to claim 4, characterized in that Based on the initial set of fuzzy line segments, perform time function fitting processing on each fuzzy line segment to obtain a set of time function parameters, including: Based on the initial set of fuzzy line segments, perform discretized resampling processing on the pixel coordinates of each fuzzy line segment to obtain a set of resampled pixel points; Based on the set of resampled pixel points, perform linear regression fitting processing on the temporal changes of the pixel points to obtain an initial set of fitted time function parameters; Based on the initial set of fitted time function parameters, perform quadratic optimization processing on the complex temporal characteristics of each fuzzy line segment to obtain an optimized set of time function parameters; Based on the optimized set of time function parameters, perform time parameter verification processing on each fuzzy line segment to obtain the set of time function parameters.

6. The redundancy design method of the 3D and AI vision sensing visible light movement according to claim 4, characterized in that Based on the representation tensor and the average image, perform shallow feature extraction and fusion to obtain preliminary fusion features, including: Perform separation processing on the representation tensor to obtain a positive-polarity event tensor and a negative-polarity event tensor; Based on the positive-polarity event tensor and the negative-polarity event tensor, perform 7×7 convolution processing on each representation tensor to obtain a positive-polarity preliminary feature map and a negative-polarity preliminary feature map; Based on the positive-polarity preliminary feature map and the negative-polarity preliminary feature map, perform connection processing in the channel dimension to obtain a merged feature map; Based on the merged feature map and the average image, perform channel attention calculation processing to obtain an attention-weighted feature map; Input the attention-weighted feature map into a preset residual block, and through short connection and convolution operations, obtain preliminary fusion features.

7. The redundant design method of the 3D and AI vision sensing visible light movement according to claim 1, characterized in that If the warning event occurs, detect the upgrade probability of the warning event being upgraded to a dangerous event according to a preset LSTM model, including: Based on the warning event, perform feature extraction processing on the features of the warning event to obtain a warning event feature vector; Based on the warning event feature vector, perform input processing on the LSTM model to obtain the output probability of the LSTM model; Based on the output probability of the LSTM model, perform threshold judgment processing on the output probability to obtain the upgrade probability; Based on the upgrade probability, perform output processing on the upgrade probability to obtain the probability of the warning event being upgraded to a dangerous event.

8. A redundant design system for a 3D and AI vision sensing visible light core module, characterized in that, The redundant design system of the 3D and AI vision sensing visible light machine core includes: An image acquisition module, configured to acquire first image data according to preset first shooting parameters; An event detection module, configured to detect whether a warning event appears in the first image data based on a preset event analysis model; A probability calculation module, configured to, if the warning event appears, detect the upgrade probability of the warning event being upgraded to a dangerous event according to a preset LSTM model, and the LSTM model is a model trained based on preset historical warning events and preset historical dangerous events; An event prevention module, configured to determine and execute warning prevention measures according to the dangerous event and a preset set of prevention measures if the escalation probability is greater than a preset escalation threshold, and to acquire second image data according to preset second shooting parameters; a danger analysis module, configured to detect the probability of occurrence of the dangerous event according to the second image data based on a preset event analysis model; The release determination module is used to stop executing the warning prevention measures if the occurrence probability is lower than a preset probability threshold.

Images