Taxi automatic top light control system based on intelligent camera

Abstract

The invention relates to the field of intelligent traffic and vehicle-mounted automatic control, in particular to a taxi automatic top light control system based on an intelligent camera, which comprises the following steps of: acquiring video stream data in a taxi in real time through the camera, and analyzing the video stream in real time by adopting a double-branch heterogeneous network structure based on a deep neural network model; carrying out multi-frame fusion filtering processing on an analysis result, and generating a passenger existence confidence coefficient curve by adopting a hybrid filtering algorithm; decision making is carried out according to the confidence coefficient curve, a dual protection mechanism is set, and when the confidence coefficient curve continuously exceeds a first preset threshold value and reaches a second preset time threshold value, a ceiling lamp state switching instruction is triggered; an instruction from the decision control module is received through a vehicle-mounted communication bus, and automatic switching of the state of the ceiling lamp is achieved in a differential signal transmission mode. The video stream is analyzed from different angles based on the deep neural network, and the accuracy of passenger state analysis is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and vehicle automatic control technology, specifically to an automatic roof light control system for taxis based on intelligent cameras. Background Technology

[0002] In traditional taxis, the status of the taxi roof light is usually controlled by a mechanical switch. After a passenger boards, the driver manually turns off the roof light, starts calculating the fare, and prints a receipt. However, with the digitalization and intelligentization of the taxi industry, order management has gradually shifted to mobile applications and digital payment systems, and mechanical switches are gradually being phased out. This necessitates a new automated solution for roof light control. Existing technologies have the following problems and drawbacks: Inability to automatically control the roof light: In the digital taxi model, since order management is entirely based on an app and there is no physical mechanical switch, the roof light status cannot be updated synchronously; Reliance on manual operation, which may lead to errors: Drivers need to manually switch the roof light status, and may fail to turn it off or on in time due to negligence, affecting operational management; Lack of a stable passenger detection mechanism: Currently, some solutions use door control signals or on-board weight sensors, but these methods are easily affected by environmental factors, leading to false alarms and delays.

[0003] This invention aims to provide an automated roof light control system based on a smart camera. By detecting whether there are passengers inside the vehicle, the system can accurately control the "empty" or "occupied" status of the taxi roof light. At the same time, a filtering algorithm is used to optimize the detection data, reduce false alarms, and improve the reliability of the system. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing an automatic roof light control system for taxis based on a smart camera.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A taxi automatic roof light control system based on an intelligent camera, comprising:

[0006] Data acquisition and passenger status analysis module: It collects in-vehicle video stream data in real time through cameras, and analyzes the video stream in real time based on a deep neural network model and a dual-branch heterogeneous network structure;

[0007] Data processing module: Performs multi-frame fusion filtering on the parsing results, adopts a hybrid filtering algorithm, performs Kalman filtering on N consecutive frames of detection data, performs exponential smoothing on discrete abrupt change points, and generates passenger presence confidence curve;

[0008] Decision control module: Makes decisions based on the confidence curve and sets up a dual protection mechanism. When the confidence curve continuously exceeds the first preset threshold and reaches the second preset time threshold, it triggers the top light state switching command.

[0009] The roof lamp control module receives instructions from the decision control module through the vehicle-mounted communication bus, adopts a differential signal transmission mode, and automatically switches the state of the roof lamp.

[0010] In a preferred embodiment, the data acquisition and passenger state analysis module adopts a high-resolution, multi-angle intelligent camera to collect real-time video stream data in the vehicle and comprehensively capture the state information of passengers, obtains multi-angle video stream data collected in real time from the vehicle-mounted camera, pre-processes the video stream, and uses a double-branch heterogeneous network structure to analyze the video stream in real time. The first branch uses a lightweight convolutional network to extract local occluded human features, and the second branch uses an attention mechanism network to capture complete human face features. The specific operation steps of the first branch to extract local occluded human features are as follows:

[0011] S1, input the pre-processed video frame into the lightweight convolutional network, and perform sliding convolution operation on the video frame through the convolution layer to extract local features in the image;

[0012] S2, reduce the dimension of the feature map through the pooling layer to reduce the calculation amount while retaining important feature information;

[0013] S3, use an activation function to perform nonlinear transformation on the output of the convolution layer;

[0014] S4, after processing by multiple convolution layers, pooling layers and activation function layers, the network outputs a group of feature vectors, which contain the feature information of the local occluded human, denoted as F body ;

[0015] The specific operation steps of the second branch to extract complete human face features are as follows:

[0016] S1, input the pre-processed video frame into the attention mechanism network, which introduces an attention unit on the basis of the convolution layer;

[0017] S2, the attention unit can highlight the features of the face part in the video frame and suppress the information of other irrelevant regions by calculating the attention weights of different regions;

[0018] S3, after multiple processing by the attention unit and the convolution layer, the network outputs a group of feature vectors, which contain the feature information of the complete human face, denoted as F face ;

[0019] The occluded area is compensated by feature-level fusion, the video frame is processed by the real-time segmentation network, the area ratio of the occluded area is calculated to obtain the value β, and the local occluded human features F body extracted by the first branch and the complete human face features F faceThe fusion process is performed using the following formula:

[0020] F fusion =β·F body +(1-β)·F face

[0021] The fused feature vector F fusion It incorporates more comprehensive passenger feature information, considering both the features of partially occluded human bodies and highlighting the features of complete faces, thus achieving occlusion compensation. This is achieved from the fused feature vector F. fusion The input is fed into a classifier, which determines the existence of passengers based on the input feature vector and outputs the probability of whether a passenger exists or not.

[0022] In a preferred embodiment, the data processing module obtains the detection result D for each frame from the data acquisition and passenger status analysis module. k When the number of consecutive received frames reaches N, Kalman filtering begins. For the k-th frame, a prediction step is first performed: based on the state estimate from the previous time step... Given the state transition matrix F, predict the state at the current time step: Based on the state covariance matrix P from the previous moment k-1|k-1 Given the state transition matrix F and the process noise covariance matrix Q, predict the state covariance matrix at the current time: P k|k-1 =F·P k-1|k-1 ·F T +Q, where P k|k-1 P represents the state estimation covariance matrix based on time k-1 at time k. k-1|k-1 The covariance matrix of the predicted state estimate at time k, F T Denotes the transpose of matrix F;

[0023] Update steps based on the predicted state: Based on the predicted state The observation matrix H and the detection result D of the current frame k Calculate the Kalman gain K k :

[0024] K k =P k|k-1 ·H T ·(H·P k|k-1 ·H T +R) -1

[0025] Among them, H T The transpose of the observation matrix H is given by the Kalman gain K. k Observation value D k and predicted state Update the state estimation of the current time:

[0026]

[0027] According to the Kalman gain K k and the predicted state covariance matrix P k|k-1 , update the state covariance matrix of the current time:

[0028] P k|k =(I-K k ·H)·P k|k-1

[0029] Where I represents the unit matrix, the state estimation value after Kalman filtering is stored in the corresponding data structure as the filtering result of the frame, for each frame of data, on the basis of Kalman filtering, check the difference between the current frame detection result D k and the filtering result S k-1 of the previous frame, when the difference exceeds the preset threshold, it is considered that the point may be a discrete mutation point, for the frame judged as a discrete mutation point, exponential moving average is used for processing:

[0030] S k =α·D k +(1-α)·S k-1

[0031] Where S k represents the result of the current frame after exponential moving average, for the frame not judged as a discrete mutation point, the result after Kalman filtering is directly taken as the final filtering result of the frame, that is, S k is equal to the state estimation value after Kalman filtering, arrange the results S k of each frame after hybrid filtering algorithm processing in time sequence to form a sequence changing with time, take the frame sequence number as the horizontal coordinate and the filtered result S k as the vertical coordinate to draw a curve, and finally generate a passenger existence confidence curve, the passenger confidence curve can reflect the change trend and confidence degree of passenger existence in a period of time, set the confidence threshold T1 and T2, T1 is used to judge the passenger existence, and T2 is used to judge the low passenger existence, when the confidence value exceeds T1 for a certain time, it is judged that the passenger is in the existing state, when the confidence value is lower than T2, it is judged that the passenger is in the non-existing state, after processing each frame, count the number of passengers in the existing state, calculate the number of elements in the recorded existing passenger list according to the list, which is the number of passengers detected in the current frame, with the playing of the video, continuously update the existing passenger list and passenger number statistical result.

[0032] In a preferred embodiment, the decision control module sets a first preset threshold Z1 for determining the effective trigger reference value of the passenger presence confidence, sets a second preset threshold Z2 for determining the reset reference value of the confidence fall-back, sets a second preset time threshold t0 for defining the effective duration of the confidence continuous overrun, obtains the passenger presence confidence curve from the data processing module in real time, which is a function of time, sets the confidence curve as a time-sequenced passenger presence confidence sequence, and performs the following judgment logic for each time-sequenced data point:

[0033] S1, compare the size relationship between the current confidence C(t) and the first preset threshold Z1;

[0034] S2, when C(t) is greater than or equal to Z1, activate the timer to accumulate the count, and when the timer reaches the second preset time threshold t0, trigger the roof light state switching instruction, and send the instruction to the roof light control module to switch the "person / empty car" state;

[0035] S3, when C(t) is less than Z1, forcibly reset the timer.

[0036] In a preferred embodiment, the roof light control module uses a differential signal transmission method to send the encoded control signal to the roof light device, uses two signal lines to transmit the signal, one line transmits the original signal, and the other line transmits the signal opposite to the original signal, sets the signal level according to the instruction requirement, sets the differential signal level to high when displaying the "person" state, and sets the differential signal level to low when displaying the "empty car" state, the roof light device receives the differential signal and decodes the signal to extract the state information, controls the internal circuit of the roof light according to the decoded state information, and realizes the display switching of the "person" or "empty car" state, after the roof light state switching is completed, the roof light control module generates state feedback information including the current roof light state, switching time content, and sends the feedback information back to the decision control module through the vehicle communication bus, so that the decision control module can confirm and record the state.

[0037] The beneficial effects of the present application are as follows:

[0038] 1. Automatic control to improve operational efficiency: automatic control is adopted to detect and automatically control the roof light state, reducing manual operation of the driver and improving the efficiency of taxi management;

[0039] 2. Improve user experience: the roof light state is switched in real time when passengers get on or off the vehicle, improve user trust and enhance the quality of taxi service;

[0040] 3. A dual-branch heterogeneous network structure based on a deep neural network, the first branch uses a lightweight convolutional network to extract local occluded human features, and the second branch uses an attention mechanism network to capture complete human face features. This structure can fully utilize the advantages of different networks, analyze the video stream from different angles, and improve the accuracy of passenger state analysis;

[0041] 4. Reduce manual intervention and optimize safety: The decision control module sets a double protection mechanism, by setting a first preset value and a second preset time value, only when the confidence curve continuously exceeds the first preset threshold value for a second preset time threshold value, the ceiling light state switching instruction is triggered, and if the confidence falls below the second threshold value during the timing process, the timer is immediately reset. This mechanism can avoid misjudgment caused by temporary confidence fluctuations, improve the accuracy and stability of decision-making, and ensure that the ceiling light state switching accurately reflects the actual situation of the passengers. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flowchart of the present application;

[0043] Figure 2 The system block diagram of the present application;

[0044] Figure 3 The AI passenger detection flowchart of the present application;

[0045] Figure 4 The ceiling light automatic control flowchart of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0047] In the description of the present application, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0048] In the description of the present application, the term "for example" is used to mean "serving as an example, instance, or illustration." Any embodiment described as "for example" in this application is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, for the purposes of explanation, numerous details are set forth in order to provide a thorough understanding. It should be apparent to those skilled in the art that the present application can be practiced without the use of these specific details. In other instances, well known structures and processes are not elaborated upon in order to avoid unnecessary detail, which can obscure the description of the present application. Thus, the present application is not intended to be limited by the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0049] As Figure 1 The present embodiment provides an intelligent camera-based automatic taxi roof light control system, comprising:

[0050] The data acquisition and passenger state analysis module acquires real-time video stream data in the vehicle through a camera, and analyzes the video stream in real time based on a deep neural network model and using a double-branch heterogeneous network structure.

[0051] In the present embodiment, the data acquisition and passenger state analysis module needs to be specifically described. The data acquisition and passenger state analysis module uses a high-resolution, multi-angle intelligent camera to acquire real-time video stream data in the vehicle, comprehensively captures the state information of passengers, acquires multi-angle video stream data in the vehicle collected in real time from the vehicle-mounted camera, pre-processes the video stream, including adjusting the resolution of the video, converting the color space of the video frame, normalizing the video frame to scale the range of pixel values to a specific interval, using a double-branch heterogeneous network structure to analyze the video stream in real time, using a lightweight convolutional network to extract local occluded human body features in the first branch, and using an attention mechanism network to capture complete facial features in the second branch. The specific operation steps of the first branch for extracting local occluded human body features are as follows:

[0052] S1, input the pre-processed video frame into the lightweight convolutional network, perform sliding convolution operation on the video frame through the convolutional layer, extract local features in the image, such as the contour of the human body and the limb features, and the output of each convolutional layer is a group of feature maps, which contain feature information of the input image at different positions and scales;

[0053] S2, reduce the dimension of the feature map through the pooling layer, reduce the calculation amount, and at the same time, retain important feature information. Through the pooling operation, the feature map can be down-sampled to make its size smaller;

[0054] S3, the output of the convolutional layer is nonlinearly transformed using an activation function, introducing a nonlinear factor, so that the network can learn more complex feature relationships;

[0055] S4, after processing by multiple convolutional layers, pooling layers and activation function layers, the network outputs a set of feature vectors, which contain the feature information of the partially occluded human body, denoted as F body ;

[0056] The specific operation steps of the second branch for extracting complete human face features are as follows:

[0057] S1, input the preprocessed video frame into the attention mechanism network, which introduces an attention unit on the basis of the convolutional layer;

[0058] S2, the attention unit can highlight the features of the human face part in the video frame and suppress the information of other irrelevant regions by calculating the attention weights of different regions, for example, by calculating the attention score of each pixel point or each position on the feature map, the attention is concentrated on the key parts of the human face;

[0059] S3, after multiple processing by the attention unit and the convolutional layer, the network outputs a set of feature vectors, which contain the feature information of the complete human face, denoted as F face ;

[0060] The occluded area is compensated by feature-level fusion, and the video frame is processed by the real-time segmentation network to calculate the area ratio of the occluded area, and the value of β is obtained, for example, the human body and background in the video frame are segmented by a semantic segmentation algorithm, then the area ratio of the occluded part in the human body region is calculated, and the local occluded human body features F body extracted by the first branch and the complete human face features F face captured by the second branch are fused according to the feature-level fusion formula, and the specific calculation formula is as follows:

[0061] F fusion =β·F body +(1-β)·F face

[0062] The fused feature vector F fusion contains more comprehensive passenger feature information, both considering the features of the partially occluded human body and highlighting the features of the complete human face, thereby realizing occlusion compensation, and the fused feature vector F fusion is input into the classifier, and the classifier judges the existence of the passenger according to the input feature vector and outputs the probability of the existence and nonexistence of the passenger.

[0063] The data processing module: the analysis result is carried out multi-frame fusion filter processing, adopts hybrid filter algorithm, executes Kalman filter to N frames of detection data in succession, executes exponential smoothing to discrete mutation point, generates passenger existence confidence curve;

[0064] In the embodiment, the data processing module needs to be specifically explained, the data processing module obtains the detection result D of each frame from the data acquisition and passenger state analysis module k , the detection result is the passenger existence probability output by the classifier, when the received continuous frame number reaches N, the Kalman filter is started to be executed, for the k frame data, first, the prediction step is carried out: according to the state estimation and the state transition matrix F of the last time, the state of the current time is predicted: According to the state covariance matrix P k-1|k-1 of the last time, the state transition matrix F and the process noise covariance matrix Q, the state covariance matrix P of the current time is predicted: k|k-1 =F·P k-1|k-1 ·F T +Q, wherein P k|k-1 indicates the covariance matrix of the state estimation of the k time based on the state estimation covariance matrix P k-1|k-1 of the k-1 time predicted at the k time, F T indicates the transpose matrix of the matrix F;

[0065] The update step is carried out according to the predicted state: according to the predicted state , the observation matrix H and the detection result D k of the current frame, the Kalman gain K k is calculated:

[0066] K k =P k|k-1 ·H T ·(H·P k|k-1 ·H T +R) -1

[0067] Wherein H T indicates the transpose matrix of the observation matrix H, according to the Kalman gain K k , the observation value D k and the predicted state , the state estimation of the current time is updated:

[0068]

[0069] According to the Kalman gain K k and the predicted state covariance matrix P k|k-1 , the state covariance matrix of the current time is updated:

[0070] P k|k = (I - K k ·H)·P k|k-1

[0071] Wherein, I represents a unit matrix, the state estimation value after Kalman filtering is stored in the corresponding data structure as the filtering result of the frame, for each frame data, on the basis of Kalman filtering, the difference between the current frame detection result D k And the filtering result S k-1 Of the previous frame is checked, when the difference exceeds the preset threshold, it is considered that the point may be a discrete mutation point, for the frame judged as a discrete mutation point, exponential moving average is used for processing:

[0072] S k = alpha D k +(1-alpha) S k-1

[0073] Wherein, S k Indicates the result of the current frame after exponential moving average, for the frame not judged as a discrete mutation point, the result after Kalman filtering is directly used as the final filtering result of the frame, that is, S k Equal to the state estimation value after Kalman filtering, the results S k Of each frame after hybrid filtering algorithm processing are arranged in time sequence to form a sequence changing with time, taking the frame sequence number as the horizontal coordinate and the filtered result S k As the vertical coordinate, a curve is drawn, and a passenger existence confidence curve is finally generated. The passenger confidence curve can reflect the change trend and confidence degree of passenger existence in a period of time. Confidence threshold T1 and T2 are set, T1 is used to judge the passenger existence, and T2 is used to judge the low passenger existence. When the confidence value exceeds T1 for a certain time, it is judged that the passenger is in the existing state, and when the confidence value is lower than T2, it is judged that the passenger is in the non-existing state. After each frame processing, the number of passengers in the existing state is counted, and the number of elements in the recorded existing passenger list is calculated. The number is the number of passengers detected in the current frame. With the playing of the video, the existing passenger list and the passenger number statistical result are continuously updated.

[0074] Decision control module: decision according to confidence curve, set double protection mechanism, when the confidence curve continuously exceeds the first preset threshold for the second preset time threshold, trigger the ceiling light state switching instruction;

[0075] In this embodiment, it needs to be specifically pointed out that the decision control module sets a first preset threshold Z1 for determining the effective trigger reference value of the passenger presence confidence, sets a second preset threshold Z2 for determining the reset reference value of the confidence falling back, and sets a second preset time threshold t0 for defining the effective duration of the confidence continuously exceeding the limit. The confidence curve of the passenger presence is obtained in real time from the data processing module, which changes with time. The confidence curve changing with time is set as a time-sequenced passenger presence confidence sequence. For each time-sequenced data point, the following judgment logic is executed:

[0076] S1, compare the size relationship between the current confidence C(t) and the first preset threshold Z1;

[0077] S2, when C(t)≥Z1, activate the timer to accumulate the count, and when the timing of the timer reaches the second preset time threshold t0, trigger the ceiling light state switching instruction, and send the instruction to the ceiling light control module to realize the switching of the "person / empty car" state;

[0078] S3, when C(t)<Z1, forcibly reset the timer, which means that even if the confidence has exceeded Z1 for a period of time before, if the confidence falls below Z2 before reaching t0, it is considered that the passenger state has not been stable enough to trigger the ceiling light switching, and the timing needs to be restarted.

[0079] Ceiling light control module: receive the instruction from the decision control module through the vehicle-mounted communication bus, adopt differential signal transmission mode, and realize automatic switching of the ceiling light state;

[0080] In this embodiment, it needs to be specifically pointed out that the ceiling light control module adopts differential signal transmission mode, sends the encoded control signal to the ceiling light device, transmits the signal using two signal lines, one line transmits the original signal, and the other line transmits the signal opposite to the original signal. According to the instruction requirement, set the signal level, when it is needed to display the "person" state, set the level of the differential signal to high level, when it is needed to display the "empty car" state, set the level of the differential signal to low level. After the ceiling light device receives the differential signal, it decodes the signal and extracts the state information. According to the decoded state information, control the internal circuit or display element of the ceiling light, realize the display switching of the "person" or "empty car" state, for example, if a high level (12V) signal is received, the indicator light representing the "person" state or the corresponding logo is lit; if a low level (0V) signal is received, the indicator light representing the "empty car" state or the corresponding logo is lit. After the ceiling light state switching is completed, the ceiling light control module generates state feedback information, including the current ceiling light state, the switching time content, and sends the feedback information back to the decision control module through the vehicle-mounted communication bus, so that the decision control module can confirm and record the state.

[0081] It should be noted that the description of the various embodiments has been presented for purposes of clarity and that it is not necessary to describe each and every embodiment separately or enumerate all its possible variations. It will be apparent to those skilled in the art that additional embodiments can be practiced which depart from the specific details of the described embodiments.

[0082] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0083] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0084] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0086] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such modifications and variations as fall within the scope of the present application.

[0087] It is apparent that those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. It is therefore intended that the present application cover all such changes and modifications that are within its scope.

Claims

1. A taxi automatic roof light control system based on an intelligent camera, characterized in that, Comprise: Data acquisition and passenger state analysis module: real-time acquisition of in-vehicle video stream data through camera, real-time analysis of video stream based on deep neural network model, using double-branch heterogeneous network structure; Data processing module: multi-frame fusion filtering processing of analysis results, using hybrid filtering algorithm, Kalman filtering for continuous N frames of detection data, exponential smoothing for discrete mutation points, generating passenger existence confidence curve; Decision control module: decision-making according to confidence curve, setting double protection mechanism, triggering ceiling light state switching instruction when confidence curve continuously exceeds first preset threshold for second preset time threshold; Ceiling light control module: receiving instructions from decision control module through vehicle-mounted communication bus, using differential signal transmission method to realize automatic switching of ceiling light state.

2. The smart camera based taxi auto roof light control system as claimed in claim 1, wherein, The data acquisition and passenger state analysis module uses high-resolution, multi-angle intelligent cameras to real-time acquire in-vehicle video stream data, comprehensively captures passenger state information, obtains real-time acquired multi-angle in-vehicle video stream data from vehicle-mounted cameras, pre-processes the video stream, uses double-branch heterogeneous network structure to real-time analyze the video stream, the first branch uses lightweight convolutional network to extract local occluded human features, and the second branch uses attention mechanism network to capture complete human face features.

3. The smart camera based taxi auto roof light control system as claimed in claim 2, wherein, The specific operation steps of the first branch extracting local occluded human features are as follows: S1, input the pre-processed video frame into the lightweight convolutional network, perform sliding convolution operation on the video frame through the convolution layer to extract local features in the image; S2, reduce the dimension of the feature map through the pooling layer to reduce the calculation amount while retaining important feature information; S3, use the activation function to perform nonlinear transformation on the output of the convolution layer; S4, after processing through multiple convolutional layers, pooling layers and activation function layers, the network outputs a set of feature vectors, which contain the feature information of the partially occluded human body, denoted as F body .

4. The smart camera based taxi auto roof light control system as claimed in claim 2, wherein, The specific operation steps of the second branch extracting complete human face features are as follows: S1, input the pre-processed video frame into the attention mechanism network, which introduces attention units based on the convolution layer; S2, the attention unit can highlight the features of the face part in the video frame and suppress the information of other irrelevant regions by calculating the attention weights of different regions; S3, after multiple processing of the attention unit and the convolution layer, the network outputs a set of feature vectors, which contain the complete facial feature information, denoted as F face .

5. The smart camera based taxi auto roof light control system as claimed in claim 4, wherein, The occlusion area is compensated by feature-level fusion, the video frame is processed by a real-time segmentation network, the area ratio of the occlusion area is calculated, the value of beta is obtained, the local occlusion human body features F body extracted by the first branch are fused with the complete face features F face captured by the second branch according to the feature-level fusion formula, and the specific calculation formula is as follows: F fusion = β · F body + (1 - β) · F face The fused feature vector F fusion The passenger feature information is more comprehensive, which considers the features of the partially occluded human body and highlights the features of the complete human face, so that the occlusion compensation is realized. From the fused feature vector F fusion The input is input into the classifier, and the classifier judges the existence of the passenger according to the input feature vector, and outputs the probability of the existence and nonexistence of the passenger.

6. The smart camera based taxi auto-roof light control system according to claim 1, wherein, The data processing module obtains the detection result D of each frame from the data acquisition and passenger state analysis module k When the number of received continuous frames reaches N, Kalman filtering is started to be performed. For the kth frame data, a prediction step is first performed: according to the state estimation and the state transition matrix F, the state at the current time is predicted: According to the state covariance matrix P k-1|k-1 , the state transition matrix F and the process noise covariance matrix Q, the state covariance matrix P k|k-1 at the current time is predicted: P k-1|k-1 · F T + Q, wherein P k|k-1 represents the covariance matrix of the state estimation at the k time based on the state estimation covariance matrix P k-1|k-1 at the k-1 time, F T represents the transpose matrix of the matrix F.

7. The smart camera based taxi auto roof light control system as claimed in claim 6, wherein, Update step from predicted state: Update from predicted state Observation matrix H and detection result D of current frame k , compute Kalman gain K k : K k = P k|k-1 · H T · (H·P k|k-1 · H T + R) -1 where H T denotes the transpose of the observation matrix H, according to the Kalman gain K k , the observation value D k and the predicted state updates the state estimation at the current time point: According to the Kalman gain K k and the predicted state covariance matrix P k|k-1 , the state covariance matrix at the current time is updated: P k|k = (I - K k · H) · P k|k-1 Wherein, I represents a unit matrix, the state estimation value after Kalman filtering is taken as the filtering result of the frame, and is stored in the corresponding data structure. For each frame of data, on the basis of Kalman filtering, the difference between the current frame detection result D k and the previous frame filtering result S k-1 is checked. When the difference exceeds a preset threshold, it is considered that the point may be a discrete mutation point. For the frame determined as a discrete mutation point, exponential smoothing average is used for processing. S k = a · D k + (1 - a) · S k-1 Wherein, S k represents the result of the current frame after exponential moving average, for the frame which is not determined as discrete mutation point, the result of Kalman filtering is directly taken as the final filtering result of the frame, that is, S k is equal to the state estimation value after Kalman filtering, the result S k of each frame after the hybrid filtering algorithm processing is arranged in time sequence to form a sequence changing with time, taking the frame sequence number as the horizontal coordinate and the filtered result S k as the vertical coordinate, a curve is drawn, and finally the passenger existence confidence curve is generated.

8. The smart camera based taxi auto roof light control system as claimed in claim 7, wherein, Set confidence thresholds T1 and T2, T1 is used to judge the existence of passengers, and T2 is used to judge the low existence of passengers, when the confidence value exceeds T1 for a certain time, it is determined that the passenger is in the existing state, when the confidence value is lower than T2, it is determined that the passenger is in the non-existing state, after processing each frame, the number of passengers in the existing state is counted, according to the recorded existing passenger list, the number of elements in the list is calculated, which is the number of passengers detected in the current frame.

9. The smart camera based taxi auto-roof light control system according to claim 1, wherein, The decision control module sets a first preset threshold Z1 to determine the effective trigger benchmark value for passenger presence confidence, sets a second preset threshold Z2 to determine the reset benchmark value for confidence decline, and sets a second preset time threshold t0 to define the effective duration of continuous confidence exceeding the limit. It acquires the passenger presence confidence curve in real time from the data processing module; this curve changes over time. The time-varying confidence curve is set as a time-series passenger presence confidence sequence. For each time-series data point, the following judgment logic is executed: S1. Compare the current confidence level C(t) with the first preset threshold Z1; S2. When C(t)≥Z1, the timer is activated to accumulate count. When the timer reaches the second preset time threshold t0, the top light state switching command is triggered and sent to the top light control module. S3. When C(t) < Z1, force reset the timer.

10. The smart camera based taxi auto-roof light control system according to claim 1, wherein, The dome light control module uses differential signal transmission to send the encoded control signal to the dome light device. Two signal lines are used: one transmits the original signal, and the other transmits the opposite signal. The signal level is set according to the command requirements. When "occupied" status is required, the differential signal level is set to high; when "vacant" status is required, the differential signal level is set to low. Upon receiving the differential signal, the dome light device decodes it, extracts the status information, and controls the internal circuitry of the dome light based on this information. After the dome light status switch is complete, the dome light control module generates status feedback information, including the current dome light status and the switching time. This feedback information is then sent back to the decision control module via the vehicle communication bus for status confirmation and recording.

Images