Intelligent security check method and device in subway station, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an intelligent security check method and device in a subway station, electronic equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, the X-ray images of all the security inspection machines are collected and sent to the background for online automatic detection, and under the condition that prohibited objects or unknown objects are detected, the X-ray images are returned to workers for real-time processing in the form of work orders, so that the security inspection process can be simplified, the security inspection efficiency is improved, the security inspection human input is reduced, and the security inspection efficiency is improved. And the security check accuracy and reliability are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of rail transit technology, and in particular to a smart security inspection method, device, electronic equipment and storage medium in a subway station. Background Art

[0002] At present, subways need to conduct security checks on passengers during daily operations. During the security check, passengers' luggage needs to be checked for prohibited items through security inspection machines. Security inspectors will check X-ray images on site to determine whether there are any illegal items in passengers' luggage. If there are suspected prohibited items, manual re-inspection will be performed to ensure the safe operation of the subway.

[0003] However, the existing security inspection process requires security personnel to view X-ray images on site. The entire process is cumbersome and complicated, and one security inspection machine needs to be equipped with at least one security inspector to view X-ray images in real time, and the manpower cost of security inspection is high. Summary of the invention

[0004] The embodiments of the present application provide a smart security inspection method, device, electronic device and storage medium in a subway station. The X-ray images of each security inspection machine are aggregated and submitted to the background online for automatic inspection. When prohibited items or unknown items are detected, they are returned to the staff in the form of a work order for real-time processing. This can simplify the security inspection process, improve security inspection efficiency, reduce security inspection manpower investment, and solve the cumbersome and complex technical problems of the security inspection process.

[0005] In a first aspect, an embodiment of the present application provides a smart security inspection method in a subway station, comprising:

[0006] Receive the target X-ray images uploaded by each security inspection machine, input the target X-ray images into a pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples, the training samples include X-ray images of specified prohibited object types;

[0007] Outputting the detection result of the target X-ray image based on the object detection model, and in the case where the detection result includes the specified prohibited item type, marking the specified prohibited item type on the target X-ray image, generating a first marked image, generating a first work order according to the first marked image, and sending the first work order to the security inspector work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to conduct a re-inspection of prohibited items;

[0008] In the case where the detection result includes an unknown object type, the unknown object type is marked on the target X-ray image, a second marked image is generated, a second work order is generated according to the second marked image, and the second work order is sent to the work terminal of the backstage staff or the on-site security personnel for manual confirmation of the unknown object type;

[0009] The processing results of the first work order and the second work order are collected regularly, annotation information of the target X-ray image is generated based on the processing results, and the object detection model is iteratively trained using the annotation information and the target X-ray image.

[0010] Furthermore, after sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes:

[0011] If the re-inspection result returned by the security inspector's work terminal is not received within the set time period, the first work order is forwarded to the subway station service system of the security inspection machine to which the target X-ray image belongs, so as to assign the task of the first work order based on the subway station service system.

[0012] Furthermore, before forwarding the first work order to the subway station service system of the security inspection machine to which the target X-ray image belongs, the method further includes:

[0013] Acquire a passenger image associated with the target X-ray image, and add the associated passenger image to the first work order.

[0014] Furthermore, the acquiring of the associated passenger image of the target X-ray image includes:

[0015] Based on the target X-ray image, call the camera in the station to detect the corresponding target detection image;

[0016] An associated target analysis is performed on the in-station surveillance video containing the target detection image to determine the associated target, and an associated passenger image of the associated target is captured from the in-station surveillance video.

[0017] Furthermore, after acquiring the associated passenger image of the target X-ray image, the method further includes:

[0018] The in-station camera of the subway station service system is called to track the target of the associated passenger based on the associated passenger image, generate a target tracking video, and send the target tracking video to the subway station service system.

[0019] Furthermore, after sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes:

[0020] Receive the re-inspection result returned by the security inspector's work terminal, and if it is determined based on the re-inspection result that there are prohibited items, collect the passenger image associated with the target X-ray image, and send the associated passenger image to each associated subway station service system for identification and early warning of illegal targets.

[0021] Furthermore, after acquiring the associated passenger image of the target X-ray image, the method further includes:

[0022] The number of violations is accumulated based on the associated passenger image, and when the number of violations reaches a set threshold, a violation target alarm of the associated passenger image is issued to each associated subway station service system.

[0023] In a second aspect, an embodiment of the present application provides a smart security inspection device in a subway station, including:

[0024] The model detection module is used to receive the target X-ray images uploaded by each security inspection machine, input the target X-ray images into a pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples, the training samples include X-ray images of specified prohibited object types;

[0025] A first re-inspection module is used to output the detection result of the target X-ray image based on the object detection model, and when the detection result contains the specified prohibited item type, mark the specified prohibited item type on the target X-ray image to generate a first marked image, generate a first work order according to the first marked image, and send the first work order to the security inspector work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to re-inspect the prohibited items;

[0026] A second re-inspection module is used to mark the unknown object type on the target X-ray image when the detection result contains an unknown object type, generate a second marked image, generate a second work order according to the second marked image, and send the second work order to the work terminal of the backstage staff or the on-site security personnel for manual confirmation of the unknown object type;

[0027] The iteration module is used to periodically collect the processing results of the first work order and the second work order, generate annotation information of the target X-ray image based on the processing results, and iteratively train the object detection model using the annotation information and the target X-ray image.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0029] memory and one or more processors;

[0030] The memory is used to store one or more programs;

[0031] When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent security inspection method in the subway station as described in the first aspect.

[0032] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the smart security inspection method in a subway station as described in the first aspect.

[0033] The embodiment of the present application receives the target X-ray image uploaded by each security inspection machine, inputs the target X-ray image into a pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples, the training samples include X-ray images of specified prohibited object types; outputs the detection result of the target X-ray image based on the object detection model, and when the detection result includes the specified prohibited object type, marks the specified prohibited object type on the target X-ray image, generates a first marked image, generates a first work order according to the first marked image, and sends the first work order to the target X-ray image. The security inspector's work terminal bound to the security inspection machine to which the X-ray image belongs is used to instruct the on-site security inspector to re-inspect prohibited items; when the detection result contains an unknown item type, the unknown item type is marked on the target X-ray image, a second marked image is generated, a second work order is generated based on the second marked image, and the second work order is sent to the backstage staff or the on-site security inspector's work terminal for manual confirmation of the unknown item type; the processing results of the first work order and the second work order are collected regularly, and the marking information of the target X-ray image is generated based on the processing results, and the object detection model is iteratively trained using the marking information and the target X-ray image. The above-mentioned technical means are adopted to summarize the X-ray images of each security inspection machine and submit them to the backstage online automatic detection, and when prohibited items or unknown items are detected, they are returned to the staff in the form of a work order for real-time processing, so as to simplify the security inspection process, improve security inspection efficiency, reduce security inspection manpower investment, and improve security inspection accuracy and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a smart security inspection method in a subway station provided in Example 1 of the present application;

[0035] Figure 2 This is a connection diagram of the backend server in Example 1 of the present application;

[0036] Figure 3 is a flowchart of the capture of the associated passenger image in the first embodiment of the present application;

[0037] Figure 4 is a schematic diagram of an associated target and an associated passenger in Embodiment 1 of the present application;

[0038] Figure 5 This is a schematic diagram of the structure of a smart security inspection device in a subway station provided in Example 2 of the present application;

[0039] Figure 6It is a structural schematic diagram of an electronic device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0041] Embodiment 1:

[0042] The intelligent security inspection method in the subway station of the present application aims to construct training samples by pre-collecting X-ray images of various types of items, especially X-ray images of designated prohibited items. Use these training samples to train the item detection model so that the model can identify prohibited items and unknown items in the X-ray images. Then, in the subsequent security inspection process, each security inspection machine in the subway station uploads the target X-ray image to the background system. The background system receives these images and inputs them into the pre-trained item detection model. The item detection model outputs the detection results of each target X-ray image.

[0043] If the detection result contains the specified prohibited item type, the system marks these prohibited items on the target X-ray image, generates a first marked image, and generates a first work order based on this image. If the detection result contains unknown item types, the system marks these unknown items on the target X-ray image, generates a second marked image, and generates a second work order based on this image.

[0044] The first work order is sent to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs, instructing the on-site security inspector to re-inspect prohibited items. The second work order is sent to the back-end staff or the on-site security inspector's work terminal to manually confirm the type of unknown items.

[0045] In addition, the system will periodically collect the processing results of the first work order and the second work order. Based on these processing results, the annotation information of the target X-ray image is generated. The object detection model is iteratively trained using this annotation information and the corresponding X-ray image to improve the accuracy and recognition ability of the model.

[0046] In this way, by automating the inspection process, the time for security inspectors to check X-ray images on site is reduced, and the efficiency of security inspection is improved. The number of security inspectors required for each security inspection machine is reduced, because the X-ray images of multiple security inspection machines can be processed uniformly by the background system. In addition, the object detection model is trained to accurately identify prohibited items and unknown items, which improves the accuracy of security inspection. By continuously collecting processing results and iterating the training model, the performance of the object detection model can be continuously optimized and improved. In addition, the automated and intelligent security inspection process helps to identify potential prohibited items more quickly and accurately, thereby enhancing the safety of subway operations.

[0047] Figure 1 A flowchart of a smart security inspection method in a subway station provided in the first embodiment of the present application is given. The smart security inspection method in the subway station provided in this embodiment can be executed by a smart security inspection device in the subway station. The smart security inspection device in the subway station can be implemented by software and / or hardware. The smart security inspection device in the subway station can be composed of two or more physical entities, or it can be composed of one physical entity. Generally speaking, the smart security inspection device in the subway station can be a computing device such as a backend server, a backend system device, a server host, etc.

[0048] The following description takes the background system as the main body of the smart security inspection method in the subway station as an example.

[0049] Reference Figure 1 The smart security inspection methods in the subway station include:

[0050] S110, receiving the target X-ray images uploaded by each security inspection machine, and inputting the target X-ray images into a pre-built object detection model. The object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples. The training samples include X-ray images of specified prohibited object types.

[0051] In the subway security inspection scenario, each security inspection machine will scan the passenger's luggage in real time. After scanning the passenger's luggage, the security inspection machine will generate X-ray images and upload these images to the background system.

[0052] like Figure 2 As shown, the backend system 111 interacts with the security inspection machines 121, the security inspection personnel work terminals 122, the subway station service system center 123, the station cameras 124, and the backend staff work terminals 112 in each subway station, and performs intelligent security inspection processing based on the collected target X-ray images, thereby improving security inspection efficiency, reducing security inspection manpower investment, and improving security inspection accuracy and reliability.

[0053] After receiving these target X-ray images, the backend system will input them into the pre-built object detection model. The object detection model is trained based on deep learning and other methods, and can automatically identify and classify objects in X-ray images.

[0054] In order to train this object detection model, training samples need to be pre-built. The training samples contain X-ray images of various types of objects, especially X-ray images of designated prohibited items. These training samples are used to train the model so that it can accurately identify prohibited items in X-ray images.

[0055] During the training process, the model learns the features of different objects and establishes a mapping relationship between the features and the object types. In this way, when a new X-ray image is input, the model can recognize and classify it based on the learned features.

[0056] The object detection model training process is a complex but critical process, which directly determines the performance and accuracy of the model in subsequent applications. The following is a detailed description of the object detection model training process:

[0057] Specifically, when training the object detection model, X-ray images of various types of objects are collected. These images cover a wide range of object categories, including common daily necessities, electronic products, and various prohibited items. The dataset can come from multiple channels, such as public datasets, data provided by cooperative institutions, or self-annotated data. Use image annotation tools (such as label Img, etc.) to annotate the collected X-ray images, divide the objects in the image into categories, and annotate the accurate location and category information for each object.

[0058] Divide the labeled data set into training set, validation set and test set. The training set is used for the model training process, the validation set is used to adjust the model's hyperparameters, and the test set is used to evaluate the model's final performance. According to the requirements of the specific task and the limitations of computing resources, you can choose a suitable deep learning model architecture, such as SSD (Single Shot MultiBox Detector), YOLO (You Only Look Once), etc. Then preprocess the training set data, including data enhancement (such as random cropping, rotation, flipping, etc.), normalization and other operations to improve the generalization ability of the model. Set the model's training parameters according to the task requirements, including learning rate, batch size, training rounds (epoch), etc.

[0059] Then use the training set data to train the model. During the training process, the model continuously learns the feature representation of the input data and adjusts the model parameters to minimize the loss function. After each training round, the validation set is used to evaluate the performance of the model, and the model's hyperparameters (such as the learning rate decay strategy) are adjusted based on the evaluation results. The performance of the trained model is evaluated using the test set. Evaluation indicators include accuracy, recall, average precision (AP), etc. Based on the performance evaluation results of the validation set and test set, the model's hyperparameters are further adjusted to optimize the model's performance. Prune and quantize the trained model to reduce the complexity and computational complexity of the model and improve the model's inference speed.

[0060] Through the above training process, a good object detection model can be trained for prohibited items detection tasks in actual scenarios such as subway security inspection.

[0061] S120. Output the detection result of the target X-ray image based on the object detection model. When the detection result includes a specified prohibited item type, mark the specified prohibited item type on the target X-ray image to generate a first marked image. Generate a first work order based on the first marked image, and send the first work order to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to re-inspect the prohibited items.

[0062] Furthermore, based on the output results of the object detection model, the smart security inspection system will further process the target X-ray image. When the detection results contain specified prohibited item types, the system will automatically mark the types of these prohibited items on the target X-ray image, thereby generating a first annotated image. This marking process clearly indicates the prohibited items that may exist in the image, making it easier for security personnel to quickly locate and identify them. The system will then generate a first work order based on the first annotated image. This work order contains detailed information about the prohibited items, such as type, location, etc., as well as instructions for re-inspection. The system will then issue the first work order to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs. In this way, on-site security personnel can receive alerts about prohibited items in a timely manner and conduct re-inspections to confirm whether prohibited items do exist.

[0063] By providing re-inspection instructions in the form of work orders when prohibited items are detected in the background, the efficiency and accuracy of security inspections have been greatly improved. Through automated labeling and work order generation, the workload of security inspectors has been reduced, while ensuring that prohibited items can be discovered and handled in a timely manner, which is of great significance for maintaining the safety and order of subway operations.

[0064] S130. When the detection result includes an unknown object type, the unknown object type is marked on the target X-ray image, a second marked image is generated, a second work order is generated according to the second marked image, and the second work order is sent to the work terminal of the backstage staff or the on-site security personnel for manual confirmation of the unknown object type.

[0065] In the actual application of the object detection model, there will be situations where the detection results contain unknown object types. In this case, the system also needs to take further measures to ensure the integrity and accuracy of the security inspection process. Among them, after the object detection model detects the target X-ray image, in addition to outputting the detection results of known object types, it is also necessary to analyze whether there are objects that the model cannot recognize, that is, unknown object types. If an unknown object type is detected, the system will automatically mark the location of these unknown objects on the target X-ray image and mark them as "unknown" or a similar identifier, thereby generating a second annotated image.

[0066] Based on the second annotated image, the system will generate a second work order. This work order will contain detailed information about the unknown object, such as location, size, shape, etc., and clearly indicate that manual confirmation is required. The system will issue the second work order to the appropriate terminal, which can be the work terminal of the back-end staff or the work terminal of the on-site security personnel, depending on the arrangement of the security inspection process and actual needs.

[0067] The staff who receives the work order will check the second annotated image and the work order content to understand the specific situation of the unknown object. The staff will manually identify the unknown object based on their own experience and knowledge by checking the original version of the target X-ray image. Once the type or nature of the unknown object is determined, the staff can record the relevant information in the system. Then, according to the type of unknown object and the requirements of the security inspection process, the staff can take further measures, such as notifying passengers, isolating items, or conducting other security checks. In this way, the second work order can strengthen communication and collaboration with on-site security personnel, ensure that the situation of unknown objects can be handled in a timely and accurate manner, and improve the efficiency and quality of security inspections.

[0068] S140. Periodically collect processing results of the first work order and the second work order, generate annotation information of the target X-ray image based on the processing results, and iteratively train the object detection model using the annotation information and the target X-ray image.

[0069] Furthermore, in order to continuously optimize the performance of the object detection model, especially in handling unknown objects or false detections, the present application regularly collects the processing results of the first work order (re-inspection results for known prohibited objects) and the second work order (confirmation results for unknown objects), and uses these results to iteratively train the model.

[0070] Among them, the cycle for collecting work order processing results is set according to actual needs, such as daily, weekly or monthly. The data of the first work order and the second work order that have been processed are automatically extracted from the system, including the original X-ray image, the annotation information in the work order (such as the location and type of prohibited items, the final confirmation result of unknown items, etc.), and the processing opinions or remarks of the security personnel. The collected work order processing results are sorted into a standard annotation format, including the location (bounding box), type (for known items) or confirmation result (for unknown items) of the items. Use image annotation tools or scripts to draw bounding boxes and labels on the original X-ray image according to the sorted annotation information to generate an image dataset with accurate annotations.

[0071] Check the generated annotated images to ensure the accuracy and consistency of the annotation information. Correct or remove images with inaccurate annotations. In order to increase the diversity of the dataset and the generalization ability of the model, appropriate data augmentation operations can be performed on the annotated images, such as rotation, scaling, cropping, etc.

[0072] Then load the current version of the object detection model and prepare the training environment, including setting the learning rate, optimizer, loss function, etc. Use the new labeled image dataset to train the model. During the training process, you can use the transfer learning method, and if possible, start training from a pre-trained model to speed up training and improve model performance. Evaluate the performance of the trained model on an independent test set, including indicators such as accuracy, recall, and average precision. Based on the evaluation results, adjust the model's hyperparameters or architecture for further optimization. If the performance of the trained model improves, replace it with a new online model for subsequent X-ray image detection tasks.

[0073] Through the above iterative training process, the object detection model can continuously learn and adapt to new environments and situations, thereby improving its accuracy and reliability in practical applications.

[0074] Optionally, after sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the application further includes:

[0075] If the re-inspection result returned by the security inspector's work terminal is not received within the set time period, the first work order is forwarded to the subway station service system of the security inspection machine to which the target X-ray image belongs, so as to assign the task of the first work order based on the subway station service system.

[0076] After the first work order is sent to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs, in order to ensure the smoothness and timeliness of the security inspection process, it is necessary to take into account the possible lack of timely feedback. Among them, the system needs to set a reasonable timeout period to determine whether the security inspector has completed the re-inspection and returned the results within the specified time. This period can be adjusted according to actual conditions, such as the busyness of the security inspection site, the average processing time of the security inspector, and other factors. The system needs to monitor the feedback from the security inspector's work terminal in real time, and record the issuance time of each first work order and the time of receiving the re-inspection results.

[0077] When the issuance time of a first work order exceeds the set timeout period, and the system has not received the re-inspection result from the security inspector's work terminal, the system needs to automatically trigger the timeout judgment logic. Once it is determined that a first work order has timed out and has not been processed, the system needs to prepare to forward the work order to the subway station service system of the security inspection machine to which the target X-ray image belongs. Specifically, it includes sorting out the work order information, marking the timeout status, and possible additional instructions or remarks. The system sends the first work order and its related information to the subway station service system through an interface or message queue. After receiving the work order, the subway station service system will assign tasks or further process them according to its own task management mechanism. The subway station service system can notify the relevant subway station service personnel or managers through internal communication tools, work emails or text messages, and inform them that there are new re-inspection tasks to be processed. The subway station service system also needs to track the progress of the assigned tasks to ensure that the tasks are processed in a timely manner. At the same time, the system also needs to promptly feedback the task processing results to the original issuing system in order to update the work order status and records.

[0078] Based on the feedback and data analysis during the task processing, the system can continuously optimize the security inspection and re-inspection process, such as adjusting the timeout period, optimizing the task assignment mechanism, and improving the response speed of security inspectors. In addition, for security inspectors' work terminals that frequently have timeouts and unprocessed situations, the background can adaptively reduce the dispatch of work orders for the work terminals. This ensures that if the security inspectors fail to complete the re-inspection task in time, the first work order can be processed promptly and effectively, thereby ensuring the smoothness and safety of the security inspection process.

[0079] Optionally, before forwarding the first work order to the subway station service system of the security inspection machine to which the target X-ray image belongs, the method further includes:

[0080] The associated passenger image of the target X-ray image is acquired, and the associated passenger image is added to the first work order.

[0081] Before forwarding the first work order to the subway station service system of the security inspection machine to which the target X-ray image belongs, in order to provide more comprehensive information and facilitate subsequent processing, the present application collects the associated passenger images of the target X-ray image and adds them to the first work order.

[0082] During the security check, there are usually in-station cameras monitoring the scene of passengers passing through the security check machine. The system can capture images of passengers from these cameras on demand. If the position of a passenger's carry-on items (such as backpacks and suitcases) in the X-ray image matches the position of the passenger in the camera, the associated passenger to whom the item in the target X-ray image belongs can be determined, and then the associated passenger image can be determined.

[0083] After determining the associated passenger images, the system needs to add these images as attachments or links to the first work order. At the same time, a brief description or explanation of the passenger image can also be added to the body of the work order. Before adding the associated passenger images to the work order, the system can perform a data consistency check to ensure that the images are consistent with the X-ray images, passenger information, etc. in terms of time, location, etc. After adding the associated passenger images, the system should update the status of the work order to indicate that the work order contains complete passenger information and is ready for the next step of processing. After completing the collection and addition of the associated passenger images, the system forwards the first work order (which already contains the associated passenger images) to the subway station service system of the security inspection machine to which the target X-ray image belongs.

[0084] By collecting and adding the relevant passenger images to the first work order, the subway station service system will be able to obtain more comprehensive information when handling the re-inspection task, which will help to more accurately judge and handle suspicious items or situations. It also provides an important basis for subsequent passenger tracking, incident investigation and other work.

[0085] Specifically, refer to Figure 3 , collect the associated passenger images of the target X-ray image, including:

[0086] S150, calling the in-station camera to detect the corresponding target detection image based on the target X-ray image;

[0087] S160: performing associated target analysis on the in-station surveillance video containing the target detection image, determining the associated target, and capturing an associated passenger image of the associated target from the in-station surveillance video.

[0088] When collecting passenger images associated with the target X-ray image, the station camera and surveillance video are used to assist in identifying and capturing the passenger image corresponding to the X-ray image.

[0089] First, based on the location and time information of the object detected in the target X-ray image, the system determines which station camera or cameras can capture passengers near the location based on the camera layout and monitoring range. At the same time, it ensures that the timestamp of the X-ray image is synchronized with the timestamp of the surveillance video of the station camera, so that the corresponding time point can be accurately found in the surveillance video.

[0090] Then, the surveillance video stream of the selected camera is called, and based on the time information in the X-ray image, the video frames at the corresponding time points are extracted as target detection images. These images will be used for subsequent target association analysis. In the target detection image, image processing or machine learning techniques (such as target detection algorithms) are used to identify possible passengers or objects. These targets should have a corresponding relationship in position with the objects in the X-ray image and are defined as associated targets.

[0091] Then, the object information in the X-ray image and the target information in the surveillance video are combined to perform correlation analysis. For example, the position of the associated target object in the X-ray image is compared with the position and action of the passenger in the surveillance video to see if they are consistent or related. Based on the results of the correlation analysis, the passengers associated with the object in the X-ray image are determined, namely the associated passengers. Figure 4 As shown, after determining the associated target a that matches the X-ray image in the monitoring video frame, an association analysis is performed based on the positions of the associated target a and the passenger b to determine that the passenger is the associated passenger.

[0092] The system will capture the video frame containing the associated passenger from the station surveillance video as the associated passenger image. These images should clearly show the passenger's facial features, body outline, and relationship with the object. Before adding the associated passenger image to the first work order, the system can perform an image quality check to ensure that the image is clear, unobstructed, and free of blur. If the image quality does not meet the requirements, other video frames can be reselected or captured.

[0093] Through the above steps, the system can accurately capture the passenger image associated with the target X-ray image and use it as part of the first work order for subsequent processing.

[0094] After acquiring the associated passenger image of the target X-ray image, it also includes:

[0095] The in-station camera of the subway station service system is called to track the target of the associated passengers based on the associated passenger images, generate the target tracking video, and send the target tracking video to the subway station service system.

[0096] After acquiring the associated passenger image of the target X-ray image, in order to further track the movement trajectory of the passenger in the subway station, the station camera of the subway station service system can be called to perform target tracking of the associated passenger and generate a target tracking video.

[0097] First, the system communicates with the in-station camera network of the subway station service system to obtain real-time video streams or historical video data. According to the information in the associated passenger image (such as the passenger's walking direction, position, etc.), select the camera that may capture the passenger and make corresponding configurations, such as adjusting the viewing angle, focal length, etc. Use image processing or machine learning technology (such as feature extraction, feature matching, etc.) to identify targets similar to the associated passenger image in the camera video stream. Once the target is identified, the system needs to use a target tracking algorithm (such as Kalman filtering, particle filtering, deep learning tracking algorithm, etc.) to continuously track the target's movement trajectory in the video.

[0098] Then, the video clips containing the associated passenger movement trajectories are edited and synthesized to generate a complete target tracking video. This video clearly shows the passenger's movement path and key behaviors in the subway station. After the target tracking video is generated, the video quality is checked to ensure that the video is smooth, without freezing or blurring. At the same time, check whether the target tracking in the video is accurate and continuous. The generated target tracking video is uploaded to the server or designated storage location, and transmitted to the subway station service system via the network.

[0099] After receiving the target tracking video, the subway station service system can perform corresponding notification and reception processing to ensure that the video can be viewed and analyzed in a timely manner. The subway station service system can conduct further passenger tracking, safety inspections or incident investigations based on the information in the target tracking video. Through the above process, the system can achieve target tracking of associated passengers in the subway station and generate target tracking videos for use by the subway station service system. It can improve the safety management level of the subway station and timely discover and deal with potential safety hazards.

[0100] Optionally, after sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes:

[0101] Receive the re-inspection results returned by the security personnel's work terminal. If it is determined that there are prohibited items based on the re-inspection results, collect the associated passenger images of the target X-ray image, and send the associated passenger images to each associated subway station service system for identification and early warning of illegal targets.

[0102] When the first work order is sent to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs, and after receiving the re-inspection result returned by the security inspector, if prohibited items are found, the system needs to take further action to ensure the safety of the subway station.

[0103] The system first verifies whether the returned re-inspection results are complete and valid, and confirms the reliability of their sources. Then the re-inspection results are analyzed to determine whether there are prohibited items. If it is confirmed that there are prohibited items, the next step of the processing flow is entered. If the associated passenger image has been collected before, the image is used directly; if it has not been collected before or the image quality does not meet the requirements, the camera in the station is called again for collection. Ensure that the collected associated passenger images are clear, unobstructed, and can accurately identify the identity and characteristics of the passengers. Transmit the associated passenger images to each associated subway station service system through a secure network channel.

[0104] The images of associated passengers should be integrated into the security monitoring and early warning module of the subway station management system. When the associated passengers appear in the field of view of other cameras in the subway station, the system should be able to identify them in real time and trigger the early warning mechanism. Based on the early warning information, the subway station management system should be able to focus on passengers carrying illegal items, realize effective monitoring and early warning of prohibited items and illegal passengers, and provide strong support for the safety management of subway stations.

[0105] Optionally, after acquiring the associated passenger image of the target X-ray image, the method further includes:

[0106] The number of violations is accumulated based on the associated passenger images. When the number of violations reaches a set threshold, a violation target alarm of the associated passenger images is sent to each associated subway station service system.

[0107] Similarly, after collecting the associated passenger images of the target X-ray image, in order to ensure the safety of the subway station and effectively manage passenger behavior, the system can accumulate the number of violations based on the associated passenger images and take corresponding alarm measures when a certain threshold is reached.

[0108] The system will store the collected images of the associated passengers and associate them with the passenger's identity information. Whenever an associated passenger is detected carrying prohibited items or committing other violations, the system will automatically record and accumulate the number of violations. The system obtains violation information in real time through the linkage capability with security inspection machines, surveillance cameras and other equipment. The system will set a reasonable violation threshold based on actual conditions and security management requirements. This threshold may be determined based on historical data and security management requirements. When the number of violations of the associated passenger reaches the set threshold, the system will trigger the violation target alarm mechanism. This mechanism will send the image and violation information of the associated passenger to each associated subway station service system so that the relevant departments can take timely measures to deal with it.

[0109] Through the above process, the subway station service system can effectively monitor and manage illegal passengers and ensure the safety and order of the subway station. At the same time, it also helps to improve passengers' safety awareness and self-awareness of compliance with regulations.

[0110] In the above, by receiving the target X-ray image uploaded by each security inspection machine, the target X-ray image is input into the pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and the model is trained based on the training samples, and the training samples include X-ray images of specified prohibited object types; the detection result of the target X-ray image is output based on the object detection model, and when the detection result includes the specified prohibited object type, the specified prohibited object type is marked on the target X-ray image, a first marked image is generated, a first work order is generated according to the first marked image, and the first work order is sent to the target X-ray image. The security personnel work terminal bound to the security inspection machine to instruct the on-site security personnel to re-inspect prohibited items; when the detection results contain unknown item types, the unknown item type is marked on the target X-ray image, a second marked image is generated, a second work order is generated based on the second marked image, and the second work order is sent to the back-end staff or the on-site security personnel's work terminal for manual confirmation of the unknown item type; the processing results of the first work order and the second work order are collected regularly, and the annotation information of the target X-ray image is generated based on the processing results, and the object detection model is iteratively trained using the annotation information and the target X-ray image. The above technical means are adopted to summarize the X-ray images of each security inspection machine and submit them to the back-end online automatic detection, and when prohibited items or unknown items are detected, they are returned to the staff in the form of a work order for real-time processing, so as to simplify the security inspection process, improve security inspection efficiency, reduce security inspection manpower investment, and improve security inspection accuracy and reliability.

[0111] Embodiment 2:

[0112] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of a smart security inspection device in a subway station provided in Example 2 of the present application. Figure 5 The smart security inspection device in the subway station provided in this embodiment specifically includes: a model detection module 21, a first re-inspection module 22, a second re-inspection module 23 and an iteration module 24.

[0113] The model detection module 21 is used to receive the target X-ray images uploaded by each security inspection machine, input the target X-ray images into the pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples, and the training samples include X-ray images of specified prohibited object types;

[0114] The first re-inspection module 22 is used to output the detection result of the target X-ray image based on the object detection model, and if the detection result contains the specified prohibited item type, mark the specified prohibited item type on the target X-ray image to generate a first marked image, generate a first work order according to the first marked image, and send the first work order to the security inspector work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to re-inspect the prohibited items;

[0115] The second re-inspection module 23 is used to mark the unknown object type on the target X-ray image when the detection result contains an unknown object type, generate a second marked image, generate a second work order according to the second marked image, and send the second work order to the work terminal of the backstage staff or the on-site security personnel for manual confirmation of the unknown object type;

[0116] The iteration module 24 is used to periodically collect the processing results of the first work order and the second work order, generate annotation information of the target X-ray image based on the processing results, and iteratively train the object detection model using the annotation information and the target X-ray image.

[0117] Specifically, after the first work order is sent to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes:

[0118] If the re-inspection result returned by the security inspector's work terminal is not received within the set time period, the first work order is forwarded to the subway station service system of the security inspection machine to which the target X-ray image belongs, so as to assign the task of the first work order based on the subway station service system.

[0119] Before forwarding the first work order to the subway station service system of the security inspection machine to which the target X-ray image belongs, it also includes:

[0120] The associated passenger image of the target X-ray image is acquired, and the associated passenger image is added to the first work order.

[0121] Acquire the associated passenger images of the target X-ray image, including:

[0122] Based on the target X-ray image, call the in-station camera to detect the corresponding target detection image;

[0123] An associated target analysis is performed on the in-station surveillance video containing the target detection image, an associated target is determined, and an associated passenger image of the associated target is captured from the in-station surveillance video.

[0124] After acquiring the associated passenger image of the target X-ray image, it also includes:

[0125] The in-station camera of the subway station service system is called to track the target of the associated passengers based on the associated passenger images, generate the target tracking video, and send the target tracking video to the subway station service system.

[0126] Specifically, after the first work order is sent to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes:

[0127] Receive the re-inspection results returned by the security personnel's work terminal. If it is determined that there are prohibited items based on the re-inspection results, collect the associated passenger images of the target X-ray image, and send the associated passenger images to each associated subway station service system for identification and early warning of illegal targets.

[0128] After acquiring the associated passenger image of the target X-ray image, it also includes:

[0129] The number of violations is accumulated based on the associated passenger images. When the number of violations reaches a set threshold, a violation target alarm of the associated passenger images is sent to each associated subway station service system.

[0130] In the above, by receiving the target X-ray image uploaded by each security inspection machine, the target X-ray image is input into the pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and the model is trained based on the training samples, and the training samples include X-ray images of specified prohibited object types; the detection result of the target X-ray image is output based on the object detection model, and when the detection result includes the specified prohibited object type, the specified prohibited object type is marked on the target X-ray image, a first marked image is generated, a first work order is generated according to the first marked image, and the first work order is sent to the target X-ray image. The security personnel work terminal bound to the security inspection machine to instruct the on-site security personnel to re-inspect prohibited items; when the detection results contain unknown item types, the unknown item type is marked on the target X-ray image, a second marked image is generated, a second work order is generated based on the second marked image, and the second work order is sent to the back-end staff or the on-site security personnel's work terminal for manual confirmation of the unknown item type; the processing results of the first work order and the second work order are collected regularly, and the annotation information of the target X-ray image is generated based on the processing results, and the object detection model is iteratively trained using the annotation information and the target X-ray image. The above technical means are adopted to summarize the X-ray images of each security inspection machine and submit them to the back-end online automatic detection, and when prohibited items or unknown items are detected, they are returned to the staff in the form of a work order for real-time processing, so as to simplify the security inspection process, improve security inspection efficiency, reduce security inspection manpower investment, and improve security inspection accuracy and reliability.

[0131] The smart security inspection device in the subway station provided in the second embodiment of the present application can be used to execute the smart security inspection method in the subway station provided in the above-mentioned first embodiment, and has corresponding functions and beneficial effects.

[0132] Embodiment three:

[0133] Embodiment 3 of the present application provides an electronic device, referring to Figure 6 The electronic device includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The number of processors in the electronic device can be one or more, and the number of memories in the electronic device can be one or more. The processor, memory, communication module, input device and output device of the electronic device can be connected via a bus or other means.

[0134] As a computer-readable storage medium, the memory can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the smart security inspection method in the subway station described in any embodiment of the present application (for example, the model detection module, the first re-inspection module, the second re-inspection module and the iteration module in the smart security inspection device in the subway station). The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0135] The communication module is used for data transmission.

[0136] The processor executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory, thereby realizing the above-mentioned smart security inspection method in the subway station.

[0137] The input device can be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device can include display devices such as display screens.

[0138] The electronic device provided above can be used to execute the smart security inspection method in the subway station provided in the above embodiment 1, and has corresponding functions and beneficial effects.

[0139] Embodiment 4:

[0140] The embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute a smart security inspection method in a subway station when executed by a computer processor. The smart security inspection method in the subway station includes: receiving target X-ray images uploaded by each security inspection machine, inputting the target X-ray image into a pre-constructed object detection model, the object detection model pre-constructs training samples based on X-ray images of various types of objects, and performs model training based on the training samples, the training samples include X-ray images of specified prohibited item types; outputting detection results of the target X-ray image based on the object detection model, and marking the specified prohibited item type on the target X-ray image when the detection results include the specified prohibited item type. , generate a first annotated image, generate a first work order based on the first annotated image, and send the first work order to the security inspector's work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to re-inspect the prohibited items; when the detection result contains an unknown item type, mark the unknown item type on the target X-ray image, generate a second annotated image, generate a second work order based on the second annotated image, and send the second work order to the backstage staff or the on-site security inspector's work terminal for manual confirmation of the unknown item type; regularly collect the processing results of the first work order and the second work order, generate the annotation information of the target X-ray image based on the processing results, and use the annotation information and the target X-ray image to iteratively train the object detection model.

[0141] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0142] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the smart security inspection method in the subway station as described above, can also execute related operations in the smart security inspection method in the subway station provided in any embodiment of the present application.

[0143] The smart security inspection device, storage medium and electronic device in the subway station provided in the above embodiments can execute the smart security inspection method in the subway station provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the smart security inspection method in the subway station provided in any embodiment of the present application.

[0144] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A smart security inspection method in a subway station, characterized in that: include: Receive target X-ray images uploaded by each security inspection machine, input the target X-ray images into a pre-built object detection model, the object detection model pre-builds training samples based on X-ray images of various types of objects, and performs model training based on the training samples, wherein the training samples include X-ray images of designated prohibited object types; Outputting the detection result of the target X-ray image based on the object detection model, and if the detection result includes the specified prohibited item type, marking the specified prohibited item type on the target X-ray image to generate a first marked image, generating a first work order according to the first marked image, and sending the first work order to a security inspector work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to conduct a re-inspection of prohibited items; In the case where the detection result includes an unknown object type, marking the unknown object type on the target X-ray image, generating a second marked image, generating a second work order according to the second marked image, and sending the second work order to a work terminal of a backstage staff or an on-site security inspector for manual confirmation of the unknown object type; The processing results of the first work order and the second work order are collected regularly, annotation information of the target X-ray image is generated based on the processing results, and the object detection model is iteratively trained using the annotation information and the target X-ray image.

2. The intelligent security inspection method in a subway station according to claim 1, characterized in that: After sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes: If the re-inspection result returned by the security inspector's work terminal is not received within the set time period, the first work order is forwarded to the subway station service system of the security inspection machine to which the target X-ray image belongs, so as to assign the task of the first work order based on the subway station service system.

3. The intelligent security inspection method in a subway station according to claim 2, characterized in that: Before forwarding the first work order to the subway station service system of the security inspection machine to which the target X-ray image belongs, the method further includes: Acquire a passenger image associated with the target X-ray image, and add the associated passenger image to the first work order.

4. The intelligent security inspection method in a subway station according to claim 3 is characterized in that: The acquiring of the associated passenger image of the target X-ray image comprises: Based on the target X-ray image, call the camera in the station to detect the corresponding target detection image; An associated target analysis is performed on the in-station surveillance video containing the target detection image to determine the associated target, and an associated passenger image of the associated target is captured from the in-station surveillance video.

5. The intelligent security inspection method in a subway station according to claim 3, characterized in that: After acquiring the associated passenger image of the target X-ray image, the method further includes: The in-station camera of the subway station service system is called to track the target of the associated passenger based on the associated passenger image, generate a target tracking video, and send the target tracking video to the subway station service system.

6. The intelligent security inspection method in a subway station according to claim 1, characterized in that: After sending the first work order to the work terminal of the security inspector bound to the security inspection machine to which the target X-ray image belongs, the method further includes: Receive the re-inspection result returned by the security inspector's work terminal, and if it is determined based on the re-inspection result that there are prohibited items, collect the passenger image associated with the target X-ray image, and send the associated passenger image to each associated subway station service system for identification and early warning of illegal targets.

7. The intelligent security inspection method in a subway station according to claim 6, characterized in that: After acquiring the associated passenger image of the target X-ray image, the method further includes: The number of violations is accumulated based on the associated passenger image, and when the number of violations reaches a set threshold, a violation target alarm of the associated passenger image is issued to each associated subway station service system.

8. A smart security inspection device in a subway station, characterized in that: include: A model detection module is used to receive the target X-ray image uploaded by each security inspection machine, input the target X-ray image into a pre-built object detection model, the object detection model pre-builds training samples based on the X-ray images of various types of objects, and performs model training based on the training samples, wherein the training samples include X-ray images of specified prohibited object types; A first re-inspection module is used to output the detection result of the target X-ray image based on the object detection model, and if the detection result includes a specified prohibited item type, mark the specified prohibited item type on the target X-ray image to generate a first marked image, generate a first work order according to the first marked image, and send the first work order to a security inspector work terminal bound to the security inspection machine to which the target X-ray image belongs, so as to instruct the on-site security inspector to re-inspect the prohibited items; A second re-inspection module is used to mark the unknown object type on the target X-ray image when the detection result contains an unknown object type, generate a second marked image, generate a second work order according to the second marked image, and send the second work order to a work terminal of a backstage staff or an on-site security inspector for manual confirmation of the unknown object type; An iteration module is used to periodically collect the processing results of the first work order and the second work order, generate annotation information of the target X-ray image based on the processing results, and iteratively train the object detection model using the annotation information and the target X-ray image.

9. An electronic device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent security inspection method in the subway station as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the intelligent security inspection method in a subway station as described in any one of claims 1-7.

Images