Security inspection machine operation method, system and apparatus, device, medium, and program product

Abstract

Provided is a security inspection machine operation method for a recognition end, relating to the field of security inspection. The method comprises: carrying out suspicious object recognition on video data from a target security inspection end; and once a suspicious object is recognized, sending a keyboard and mouse message to the target security inspection end on the basis of a preset operation instruction, wherein the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message. Further provided are a security inspection machine operation system and apparatus, a device, a storage medium, and a program product.

Description

Security inspection machine operation methods, systems, devices, equipment, media and program products

[0001] This application claims priority to Chinese patent application No. 202311027711.3 filed on August 15, 2023, the contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of security inspection, and more specifically, to a method, system, device, equipment, medium and program product for operating a security inspection machine. Background Art

[0003] In the security inspection industry, data formats and software systems of security inspection machines from different manufacturers are incompatible. Providing artificial intelligence (AI) recognition services for third-party security inspection machines may not be possible directly on the machines themselves. Instead, independent AI recognition devices can be provided for each third-party machine to implement AI recognition capabilities.

[0004] Because the data formats or software business systems of different manufacturers are incompatible with each other, and the communication interfaces and protocols of the third-party manufacturers' security inspection machines and AI recognition devices may be different, or the third-party manufacturers encrypt the communication interfaces, the AI ​​recognition device cannot directly operate one or more third-party manufacturers' security inspection machines through the network protocol to handle them in a timely manner after identifying suspicious objects.

[0005] Summary of the Invention

[0006] One aspect of an embodiment of the present disclosure provides a security inspection machine operation method for an identification terminal, comprising: identifying suspicious objects on video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by the security inspection machine of the target security inspection terminal, the target security inspection terminal being any one of N security inspection terminals, and N being an integer greater than or equal to 1; after identifying a suspicious object, sending a keyboard and mouse message to the target security inspection terminal based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection terminal is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0007] According to an embodiment of the present disclosure, before sending the keyboard and mouse message to the target security inspection end based on the preset operation instruction, the method also includes: monitoring the keyboard and mouse message generated locally by the identification end; after monitoring the keyboard and mouse message, obtaining the current operation mode of the identification end to determine whether to send the keyboard and mouse message.

[0008] According to an embodiment of the present disclosure, the current operating mode includes a first operating mode or a second operating mode. Before determining the current operating mode of the identification end, the method further includes: switching to the first operating mode in response to a first mode instruction, wherein the identification end is configured to allow the sending of the keyboard and mouse messages in the first operating mode; and / or switching to the second operating mode in response to a second mode instruction, wherein the identification end is configured to prohibit the sending of the keyboard and mouse messages in the second operating mode, so as to operate the identification end locally.

[0009] According to an embodiment of the present disclosure, the identification end includes a keyboard and mouse component, and the keyboard and mouse component includes a keyboard and / or a mouse. The sending of keyboard and mouse messages to the target security inspection end based on preset operation instructions includes: in the first operation mode, sending the automatically generated keyboard and mouse messages to the target security inspection end, and / or the keyboard and mouse messages manually input by the user operating the keyboard and mouse component.

[0010] According to an embodiment of the present disclosure, the method further includes: in the second operation mode, operating the identification terminal locally in response to the automatically generated keyboard and mouse information and / or the manually input keyboard and mouse message.

[0011] According to an embodiment of the present disclosure, the identification end includes an identification computer and a keyboard and mouse simulation component, the target security inspection end includes a target security inspection computer connected to the security inspection machine, and the method further includes: providing the keyboard and mouse simulation component to connect the identification computer and the target security inspection computer; wherein, sending the keyboard and mouse message to the target security inspection end includes: after the identification computer identifies a suspicious object, the keyboard and mouse simulation component sends the keyboard and mouse message to the target security inspection computer through the keyboard and mouse simulation component, and the keyboard and mouse simulation component is used to simulate the keyboard message and / or mouse message of the target security inspection computer.

[0012] According to an embodiment of the present disclosure, before performing suspicious object identification on the video data from the target security inspection end, the method further includes: acquiring the video signal of the target security inspection computer to obtain the video data; wherein the identification computer is connected to a display, and the suspicious object identification includes: performing suspicious object identification on the image frame of the object to be inspected in the video data; and outputting the suspicious object identification conclusion and the suspicious object label with the video data to the display for superimposed display.

[0013] According to an embodiment of the present disclosure, the superimposed display includes: superimposed display according to video parameters during acquisition, wherein the video parameters include a video resolution and a frame rate of the video signal.

[0014] Another aspect of the present disclosure provides a security inspection machine operating system, including:

[0015] an identification computer connected to N security inspection computers, configured to execute the method described in any one of the above items, where N is an integer greater than or equal to 1;

[0016] An external device is connected to the identification computer, wherein the external device includes at least one of a keyboard, a mouse, or the display, wherein at least one target security inspection computer among the N security inspection computers is configured to share the external device with the identification computer.

[0017] Another aspect of the present disclosure provides a security inspection machine operating device for use at an identification terminal, including:

[0018] An object recognition module, configured to identify suspicious objects based on video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection terminal, where the target security inspection terminal is any one of N security inspection terminals, where N is an integer greater than or equal to 1;

[0019] The keyboard and mouse message module is used to send keyboard and mouse messages to the target security inspection end based on preset operation instructions after identifying a suspicious object, wherein the keyboard and mouse messages include keyboard messages and / or mouse messages, and the target security inspection end is configured to execute the preset operation instructions in response to the keyboard and mouse messages.

[0020] The security inspection machine operating device includes modules for respectively executing each step of any one of the methods described above.

[0021] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method described above.

[0022] Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method described above.

[0023] Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0025] FIG1 schematically shows an application scenario of a method for operating a security inspection machine according to an embodiment of the present disclosure;

[0026] FIG2 schematically shows a flow chart of a method for operating a security inspection machine according to an embodiment of the present disclosure;

[0027] FIG3 a schematically shows a flow chart of determining a current operation mode according to an embodiment of the present disclosure;

[0028] FIG3 b schematically shows an architecture diagram of a keyboard and mouse switch according to an embodiment of the present disclosure.

[0029] FIG4 schematically shows a flow chart of sending keyboard and mouse messages according to an embodiment of the present disclosure;

[0030] FIG5 schematically shows an application scenario diagram of a method for operating a security inspection machine according to another embodiment of the present disclosure;

[0031] FIG6 schematically shows a flow chart of overlay display according to an embodiment of the present disclosure;

[0032] FIG7 schematically shows a schematic diagram of separate display of video data according to an embodiment of the present disclosure;

[0033] FIG8 schematically shows a schematic diagram of video data overlay display according to an embodiment of the present disclosure;

[0034] FIG9 schematically shows a structural block diagram of a security inspection machine operating device according to an embodiment of the present disclosure; and

[0035] FIG10 schematically shows a block diagram of an electronic device suitable for implementing a method for operating a security inspection machine according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to facilitate understanding of the technical solutions of the embodiments of the present disclosure, some technical terms involved in the present disclosure are first introduced.

[0037] Identification end: includes hardware and software that implement AI recognition functions.

[0038] Security inspection end: includes hardware and software for implementing security inspection functions.

[0039] Security inspection machine: belongs to the security inspection end, including scanning equipment and security inspection computer. For example, the scanning equipment may include X-ray machine, conveyor belt and security inspection channel.

[0040] Security inspection computer: includes a computer host, which is connected to the scanning device to receive scanning data to form a video signal, or issue scanning device control instructions.

[0041] Security inspection machine operating system: belongs to the identification end, including the identification computer, keyboard and mouse simulation components and their external devices and other hardware and software systems.

[0042] Identify the computer: including the computer host. The computer host refers to the main body part of the computer excluding peripherals such as input and output devices.

[0043] Keyboard messages: include key messages and character messages. The underlying operating system (such as Windows or Linux) can generate keyboard messages in response to application instructions (such as a virtual keyboard, or calling the keyboard interface to directly input instructions), keyboard input signals, touch input, voice control, etc.

[0044] Mouse messages: Mouse movements, clicks, and scroll wheel actions generate mouse messages, which are generated by data sent from the mouse's internal MCU to the host. Mouse messages can also be generated by operating the mouse through keyboard, touch input, voice control, and other methods.

[0045] Preset operation instructions: instructions that can affect the action or status of the security inspection machine, such as the emergency stop instruction, start instruction, alarm instruction, conveyor belt forward instruction, acceleration and deceleration instruction of the scanning equipment.

[0046] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0047] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0048] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0049] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0050] To provide AI-powered recognition services for security inspection machines from different manufacturers, intelligent analysis can be performed on the video streams connected to the machines. By structuring the video packets within the streams, AI-powered recognition can be performed in real time, providing AI-powered conclusions to image judges. This video access method provides intelligent image review services for security inspection machines from different manufacturers, offering high compatibility and avoiding technical barriers caused by incompatibility issues with raw data formats from different manufacturers. However, there remains the issue of AI-powered recognition devices being unable to directly control third-party security inspection machines through network protocols to promptly address suspicious objects.

[0051] The embodiments of the present disclosure provide a security inspection machine operation method, system, device, equipment, medium and program product, which is different from the method of directly operating the security inspection machines of third-party manufacturers through network protocols. It can overcome obstacles such as different communication interfaces and protocols, incompatible data formats or encryption of communication interfaces by third-party manufacturers. After identifying suspicious objects in the video data of the target security inspection end, the target security inspection machine end of the third-party manufacturer can promptly receive and identify preset operation instructions in the form of keyboard and mouse messages, so that the target security inspection end executes the preset operation instructions to achieve specific operations.

[0052] Figure 1 schematically illustrates an application scenario of a method for operating a security inspection machine according to an embodiment of the present disclosure. However, the present disclosure is not limited to the application scenario and architecture shown in Figure 1 .

[0053] As shown in Figure 1, application scenario 100 may include an identification terminal and a security inspection terminal. The identification terminal may include a security inspection machine operating system 110. The security inspection terminals may include security inspection terminals 1-121, security inspection terminals 2-122, and so on, security inspection terminals N-12N. Security inspection machine operating system 110 may include an identification computer 111 and an external device 112. Security inspection machine operating system 110 may be communicatively connected to at least one of security inspection terminals 1-N, for example, by directly connecting via identification computer 111 or by providing other hardware between identification computer 111 and security inspection terminals 1-N to achieve this connection.

[0054] In some embodiments, each of the security inspection terminals 1 to N includes a security inspection computer. The identification computer 111 can be connected to N security inspection computers to execute the security inspection machine operating method provided in one or more embodiments of the present disclosure, where N is an integer greater than or equal to 1. An external device 112 is connected to the identification computer 111 and includes at least one of a keyboard, a mouse, or a display. At least one target security inspection computer among the N security inspection computers is configured to share an external device with the identification computer.

[0055] In one or more embodiments of the present disclosure, in addition to the security inspection computer and the identification computer set up locally at the security inspection site, at least one of the security inspection computer and the identification computer may be replaced by a cloud host set up remotely based on cloud computing technology.

[0056] According to the embodiments of the present disclosure, the security inspection computer and the identification computer share external devices, which can avoid the confusion problem caused by multiple sets of external devices, save external device costs and equipment layout space, and improve the operational convenience of multiple security inspection machines.

[0057] It should be understood that the present disclosure is not intended to limit the solution to relying on the target security inspection computer and the identification computer sharing external devices to overcome the shortcomings of the related art. For example, each security inspection computer may have its own matching external devices, or at least two security inspection computers may share a set of external devices. This configuration can be tailored to the specific situation.

[0058] The following will describe in detail the method for operating the security inspection machine according to an embodiment of the present disclosure based on the scenario described in FIG1 through FIG2 to FIG8 .

[0059] FIG2 schematically shows a flow chart of a method for operating a security inspection machine according to an embodiment of the present disclosure.

[0060] As shown in FIG2 , the security inspection machine operation method of this embodiment is used at the identification end, and includes operations S210 to S230 .

[0061] In operation S210, suspicious objects are identified on video data from a target security inspection terminal, where the video data is obtained by scanning an object to be inspected by a security inspection machine at the target security inspection terminal. The target security inspection terminal is any one of N security inspection terminals, where N is an integer greater than or equal to 1.

[0062] For example, objects to be inspected may include luggage, parcels, packaging bags, and other items contained therein, as well as individuals and their belongings. The security inspection machine may employ one or more scanning methods, including X-ray, millimeter wave, or visible light. The recognition terminal can obtain video data from the target security inspection terminal in real time or asynchronously via wired or wireless means. Similarly, it can also perform real-time or asynchronous suspicious object identification. Suspicious objects include prohibited items such as weapons, ammunition, highly toxic substances, or flammable and explosive materials, as well as other objects specified by the public place management.

[0063] In some embodiments, the identification process may include: obtaining an X-ray image of the package to be inspected; performing object detection on the X-ray image based on pre-stored templates in a contour matching template library, and obtaining a suspicious object identification conclusion. The identification process may also include: inputting the X-ray image into an artificial intelligence model, such as a classification model implemented based on a machine learning algorithm, to obtain a suspicious object identification conclusion. The identification process may also include: installing a camera above the X-ray machine entrance and installing identification strips on both sides of the conveyor belt; and performing object detection on the visible light image and the X-ray scan image using a deep focused convolutional residual network.

[0064] In operation S220, after a suspicious object is identified, a keyboard and mouse message is sent to the target security inspection end based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0065] In some embodiments, the recognition end can simultaneously recognize video data from one or more target security inspection ends, and send keyboard and mouse messages to the corresponding target security inspection end after recognizing a suspicious object.

[0066] For example, there may be multiple preset operation instructions, for example, the corresponding preset operation instruction is determined according to the category of the suspicious object, and each preset operation instruction has a corresponding keyboard and mouse message. The recognition end can use a wired connection or a wireless connection to send the keyboard and mouse message to the target security inspection end.

[0067] It can be understood that after the target security inspection end receives the keyboard and mouse message, the security inspection machine believes that it has received the keyboard and mouse message generated by an external device or locally, so it can recognize the keyboard and mouse message and execute the corresponding preset operation instructions.

[0068] In some embodiments, the recognition computer may generate keyboard and mouse messages locally, or may accept user input, such as keyboard and mouse input, or voice control input, touch input, or gesture input to generate keyboard and mouse messages.

[0069] According to the embodiments of the present disclosure, unlike the method of directly operating the security inspection machines of various third-party manufacturers through network protocols, it is possible to overcome obstacles such as different communication interfaces and protocols, incompatible data formats, or encryption of communication interfaces by third-party manufacturers. After a suspicious object is identified in the video data of the target security inspection end, the target security inspection machine end of the third-party manufacturer can promptly receive and identify the preset operation instructions in the form of keyboard and mouse messages, thereby enabling the target security inspection end to execute the preset operation instructions to achieve specific operations.

[0070] FIG3 a schematically shows a flow chart of determining a current operation mode according to an embodiment of the present disclosure.

[0071] Before sending the keyboard and mouse message to the target security inspection terminal based on the preset operation instruction, as shown in FIG3 a , this embodiment includes operations S310 to S320 .

[0072] In operation S310, keyboard and mouse messages generated locally by the recognition terminal are monitored.

[0073] For example, deploy an application to monitor the keyboard and mouse events of the Windows system, and implement monitoring and identification of the underlying keyboard and mouse messages.

[0074] In operation S320, after the keyboard and mouse message is monitored, the current operation mode of the recognition terminal is obtained to determine whether to send the keyboard and mouse message.

[0075] Exemplarily, the operation mode is used to set whether to execute the forwarding action of the keyboard and mouse message.

[0076] According to an embodiment of the present disclosure, the sending of the keyboard and mouse message is further confirmed through the current operation mode, thereby avoiding erroneous operation on the target security inspection terminal when the keyboard and mouse message does not need to be sent.

[0077] In some embodiments, the current operation mode includes a first operation mode or a second operation mode, and before determining the current operation mode of the identification terminal, the method further includes: switching to the first operation mode in response to a first mode instruction, wherein the identification terminal is configured to allow sending keyboard and mouse messages in the first operation mode, and / or switching to the second operation mode in response to a second mode instruction, wherein the identification terminal is configured to prohibit sending keyboard and mouse messages in the second operation mode, so as to locally operate the identification terminal.

[0078] For example, in the first operating mode, the forwarding action is executed, while in the second operating mode, the forwarding action is stopped. The first mode instruction or the second mode instruction may include one or more preset shortcut key instructions, voice instructions, or gesture instructions. The local recognition end may include software and hardware such as the recognition computer and its applications.

[0079] In some embodiments, when the security inspection computer needs to be operated, a pre-set shortcut key is used to switch, and the recognition computer completely forwards the collected mouse and keyboard messages to the security inspection computer. When operating the recognition computer, only the pre-set shortcut key needs to be used to switch, and all messages are not forwarded or distributed to the security inspection computer.

[0080] In the related art, the security inspection computer and the identification computer are two different computers, and each needs to be provided with a set of mouse and keyboard sets. One set of keyboard and mouse is needed to operate the security inspection machine, and another set of keyboard and mouse is needed to operate the identification computer, which brings great inconvenience to on-site personnel. The two sets of mouse and keyboard are easily confused on the same operating table.

[0081] FIG3 b schematically shows an architecture diagram of a keyboard and mouse switch according to an embodiment of the present disclosure.

[0082] In some embodiments, as shown in FIG3b , a separate keyboard and mouse switch can be provided to allow security inspection computers 1-N and the identification computer to share a single mouse and keyboard, thus preventing the confusion of multiple mouse and keyboard pairs on a single console. The keyboard and mouse switch uses different shortcut keys to synchronously transmit keyboard and mouse signals to each controlled computer, enabling operation of a single or multiple controlled computers and meeting keyboard and mouse switching requirements.

[0083] However, the keyboard and mouse switch also has certain defects. Each computer is independent of each other and passively receives key and mouse signals from the keyboard and mouse switch. If the recognition computer identifies high-risk items, it needs to actively provide emergency stop services or personalized notification services, but the keyboard and mouse switch cannot meet this business need.

[0084] According to the embodiments of the present disclosure, in a first operating mode, keyboard and mouse messages are forwarded to the target security inspection terminal, enabling operations on the target security inspection terminal. In a second operating mode, keyboard and mouse messages are not forwarded to the target security inspection terminal, enabling local operations. Thus, while the target security inspection terminal and the identification terminal can share a set of mouse and keyboard, keyboard and mouse messages can be used to enable the target security inspection terminal of a third-party manufacturer to promptly receive and recognize preset operation instructions, thereby causing the target security inspection terminal to execute the preset operation instructions to achieve specific operations.

[0085] In some embodiments, the identification end includes a keyboard and mouse component, and the keyboard and mouse component includes a keyboard and / or a mouse. In operation S220, sending a keyboard and mouse message to the target security inspection end based on a preset operation instruction includes: in a first operation mode, sending an automatically generated keyboard and mouse message to the target security inspection end, and / or a keyboard and mouse message manually input by the user operating the keyboard and mouse component.

[0086] For example, the keyboard can include a universal keyboard with an A-Z alphabetical area, a 0-9 numerical area, and other symbol areas, or a customized keyboard adapted to the functions of the identification terminal or the target security inspection terminal. Similarly, the mouse can also be universal or customized.

[0087] According to the embodiments of the present disclosure, in security inspection scenarios, the time it takes for each package to be inspected is roughly the time it takes between the input and output of the conveyor belt. Therefore, this time is limited. After a suspicious object is identified, keyboard and mouse messages are automatically generated to operate the target security inspection terminal, achieving high timeliness. Furthermore, through the keyboard and mouse components, users can set and operate the target security inspection terminal according to their preferences, providing more operation methods and improving convenience.

[0088] In some embodiments, the further embodiment includes: in a second operation mode, operating the identification terminal locally in response to automatically generated keyboard and mouse information and / or manually input keyboard and mouse messages.

[0089] According to an embodiment of the present disclosure, when operating the identification computer, switching is achieved according to the pre-set second mode instructions, and all messages are not forwarded or distributed to the security inspection computer, thereby enabling automated operation and / or manual operation, thereby improving the operational convenience of the identification terminal.

[0090] Figure 4 schematically shows a flowchart of sending keyboard and mouse messages according to an embodiment of the present disclosure. Figure 5 schematically shows an application scenario diagram of a method for operating a security inspection machine according to another embodiment of the present disclosure.

[0091] As shown in Figure 4, the sending of keyboard and mouse messages in this embodiment includes operations S410 to S420. Operation S420 is an embodiment of operation S220. The identification end includes an identification computer and a keyboard and mouse simulation component.

[0092] In operation S410, a keyboard and mouse simulation component is provided to connect the identification computer and the target security inspection computer.

[0093] Exemplarily, the keyboard and mouse simulation component may include keyboard and mouse simulation hardware and / or keyboard and mouse simulation software. In some embodiments, the hardware may be a wired connection. In other embodiments, the hardware may be a wireless connection, for example, by inserting a converter module into the identification computer and the target security inspection computer, respectively, so that the two converter modules can be wirelessly connected. In other embodiments, the software may virtualize one or more serial ports in the computer, for example, virtualizing serial ports in the identification computer and the target security inspection computer that can directly communicate with each other, or setting up a serial port server to implement keyboard and mouse message forwarding from the identification computer to the target security inspection computer.

[0094] In operation S420, after identifying a suspicious object, the identification computer sends keyboard and mouse messages to the target security inspection computer through the keyboard and mouse simulation component, where the keyboard and mouse simulation component is used to simulate keyboard messages and / or mouse messages of the target security inspection computer.

[0095] For example, the keyboard and mouse simulation component can provide a virtual serial port to convert data generated by the identification computer, thereby simulating keyboard and / or mouse messages. For example, a WG Wiegand to USB keyboard converter can be used to simulate a keyboard and virtual serial port. As shown in Figure 5, in the security inspection computer 1 at the security inspection terminal 1, the virtual serial port is recognized as a real port by the operating system and serial application. Input from the virtual serial port can be simulated as keyboard input, thereby generating recognizable keyboard messages on the security inspection computer, enabling communication between the identification terminal and the target security inspection terminal.

[0096] According to the embodiment of the present disclosure, the security inspection computer and the identification computer are two different computers, and the keyboard and mouse simulation components realize keyboard and mouse message routing and forwarding. By pre-setting mode instructions (such as shortcut keys), a shared set of keyboard and mouse components is realized, which brings convenience to the user's operation.

[0097] Figure 6 schematically shows a flow chart of superimposed display according to an embodiment of the present disclosure. Figure 7 schematically shows a schematic diagram of separate display of video data according to an embodiment of the present disclosure. Figure 8 schematically shows a schematic diagram of superimposed display of video data according to an embodiment of the present disclosure.

[0098] As shown in Figure 6, the overlay display of this embodiment includes operations S610 to S630, wherein it is recognized that the computer is connected to a display, and operations S620 and S630 are embodiments of operation S210.

[0099] In operation S610 , a video signal of a target security inspection computer is collected to obtain video data.

[0100] In operation S620, suspicious object recognition is performed on the image frame of the object to be detected in the video data.

[0101] Exemplarily, the image frame is an image containing an object to be inspected, which can be subjected to AI recognition.

[0102] In operation S630 , the suspicious object recognition conclusion, the suspicious object label, and the video data are output to a display for superimposed display.

[0103] For example, the suspicious object recognition conclusion includes a suspicious object existence conclusion or a normal conclusion, and the suspicious object annotation may include suspicious object category information and a standard frame for selecting the suspicious object in the image.

[0104] In some embodiments, the display is overlaid based on the video parameters at the time of acquisition, where the video parameters include the video resolution and frame rate of the video signal.

[0105] 5 , the recognition computer 111 collects the video signal (VGA port, DVI port, or HDMI port, etc.) of the security inspection computer 1 and collects the video data in its original resolution and frame rate. The recognition computer 111 performs video object tracking analysis, package segmentation, and other processing on each frame of the collected video data. At the same time, AI intelligent recognition is performed, and the suspect conclusion identified by the AI ​​is labeled as a suspect object on the video layer of the original video frame. The superimposed video image is displayed and output to the monitor in accordance with the original collected video resolution and frame rate.

[0106] Users can check for items of particular interest, such as explosives, firearms, and controlled knives, based on their preferences. If any of these items are detected during security checks, a designated keyboard message can be sent to the security computer 1 to stop the device or issue a warning. As shown in Figure 5, the keyboard and mouse are connected to the recognition computer 111, which intercepts keyboard and mouse messages and distributes them to the security computer 1 based on business requirements. Keyboard and mouse message transmission can be customized to achieve automatic emergency stops or warnings, providing a method for interaction between different computers.

[0107] Illustratively, the display screen of the security inspection computer 1 connected to the monitor is shown in FIG7 , the recognition computer 111 collects the video stream of the security inspection computer 1 , and then superimposes the display result on the monitor. The display screen is shown in FIG8 , which superimposes the annotation box and conclusion on the basis of FIG7 .

[0108] According to the embodiment of the present disclosure, the security inspection computer 1 and the identification computer 111 can share the same display screen and mouse and keyboard, thereby avoiding confusion and interference caused by multiple display screens to the reviewers and reducing the cost of deploying multiple display screens.

[0109] According to the embodiments of the present disclosure, displaying according to the originally acquired video parameters can ensure the overlay display effect of the video data on the display, avoiding distortion such as an overly large annotation box, elongated wrapped image, or missing part of the image.

[0110] Based on the above-mentioned security inspection machine operating method, the present disclosure further provides a security inspection machine operating device, which will be described in detail below in conjunction with FIG9 .

[0111] FIG9 schematically shows a structural block diagram of a security inspection machine operating device according to an embodiment of the present disclosure.

[0112] As shown in FIG. 9 , the security inspection machine operating device 900 of this embodiment includes an object recognition module 910 and a keyboard and mouse message module 920 .

[0113] The object recognition module 910 can perform operation S210 to identify suspicious objects in the video data from the target security inspection end, where the video data is obtained by scanning the object to be inspected by the security inspection machine of the target security inspection end. The target security inspection end is any one of N security inspection ends, where N is an integer greater than or equal to 1.

[0114] The keyboard and mouse message module 920 can execute operation S220, which is used to send a keyboard and mouse message to the target security inspection end based on the preset operation instruction after identifying a suspicious object, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0115] In some embodiments, the security inspection machine operating device 900 may further include a mode determination module, which may execute operations S310 to S320, which will not be described in detail here.

[0116] In some embodiments, the security inspection machine operating device 900 may further include a mode switching module for switching to a first operating mode in response to a first mode instruction, wherein the identification end is configured to allow the sending of keyboard and mouse messages in the first operating mode; and / or switching to a second operating mode in response to a second mode instruction, wherein the identification end is configured to prohibit the sending of keyboard and mouse messages in the second operating mode, so as to operate the identification end locally.

[0117] In some embodiments, the keyboard and mouse message module 920 is further configured to, in a first operating mode, send automatically generated keyboard and mouse messages and / or keyboard and mouse messages manually input by a user operating a keyboard and mouse component to a target security inspection terminal. Alternatively, in a second operating mode, the module is configured to locally operate the identification terminal in response to automatically generated keyboard and mouse messages and / or manually input keyboard and mouse messages.

[0118] In some embodiments, a keyboard and mouse simulation component is provided to connect the identification computer and the target security inspection computer. The keyboard and mouse message module 920 is also used to enable the identification computer to send keyboard and mouse messages to the target security inspection computer through the keyboard and mouse simulation component after identifying a suspicious object. The keyboard and mouse simulation component is used to simulate the keyboard messages and / or mouse messages of the target security inspection computer.

[0119] In some embodiments, the security inspection machine operating device 900 may further include a video acquisition module for acquiring a video signal of a target security inspection computer to obtain video data (operation S610). The object recognition module 910 is also used to perform operations S620 to S630, which will not be described in detail here.

[0120] In some embodiments, the object recognition module 910 is further configured to overlay and display based on video parameters during acquisition, where the video parameters include a video resolution and a frame rate of the video signal.

[0121] It should be noted that the security inspection machine operating device 900 includes modules for executing each step of any of the embodiments described in Figures 2 to 7 above. The implementation methods, technical problems solved, functions achieved, and technical effects achieved of each module / unit / subunit in the device embodiments are the same or similar to the implementation methods, technical problems solved, functions achieved, and technical effects achieved of each corresponding step in the method embodiments, and are not further described here.

[0122] According to an embodiment of the present disclosure, any multiple modules in the object recognition module 910 and the keyboard and mouse message module 920 can be combined into a single module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module.

[0123] According to an embodiment of the present disclosure, at least one of the object recognition module 910 and the keyboard and mouse message module 920 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the object recognition module 910 and the keyboard and mouse message module 920 can be at least partially implemented as a computer program module, which can perform the corresponding function when executed.

[0124] FIG10 schematically shows a block diagram of an electronic device suitable for implementing a method for operating a security inspection machine according to an embodiment of the present disclosure.

[0125] As shown in Figure 10, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0126] Various programs and data required for the operation of the electronic device 1000 are stored in the RAM 1003. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. The processor 1001 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 1002 and / or the RAM 1003. It should be noted that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0127] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 1010 as needed so that a computer program read therefrom can be installed into the storage portion 1008 as needed.

[0128] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments. Alternatively, the computer-readable storage medium may exist independently, without being incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the methods according to the embodiments of the present disclosure.

[0129] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0130] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiments of the present disclosure.

[0131] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0132] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0133] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0134] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0135] One or more of the above-mentioned embodiments have the following beneficial effects: unlike the method of directly operating the security inspection machines of various third-party manufacturers through network protocols, it can overcome obstacles such as different communication interfaces and protocols, incompatible data formats, or encryption of communication interfaces by third-party manufacturers. After identifying suspicious objects in the video data of the target security inspection end, without changing the business system of the target security inspection end, the target security inspection machine of the third-party manufacturer can promptly receive and identify preset operation instructions in the form of keyboard and mouse messages, thereby enabling the target security inspection end to execute the preset operation instructions to achieve specific operations.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or couplings are intended to fall within the scope of this disclosure.

[0138] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for operating a security inspection machine, used for an identification terminal, comprising: Perform suspicious object identification on video data from a target security inspection end, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection end, the target security inspection end being any one of N security inspection ends, where N is an integer greater than or equal to 1; After identifying a suspicious object, a keyboard and mouse message is sent to the target security inspection end based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message.

2. The method according to claim 1, wherein: Before sending a keyboard and mouse message to the target security inspection terminal based on a preset operation instruction, the method further includes: Monitor the keyboard and mouse messages generated locally by the recognition terminal; After monitoring the keyboard and mouse message, the current operation mode of the identification terminal is obtained to determine whether to send the keyboard and mouse message.

3. The method according to claim 2, wherein: The current operation mode includes a first operation mode or a second operation mode. Before determining the current operation mode of the identification terminal, the method further includes: In response to a first mode instruction, switching to the first operation mode, wherein the identification end is configured to allow sending the keyboard and mouse message in the first operation mode; and / or In response to the second mode instruction, the second operation mode is switched, wherein the identification end is configured to prohibit sending the keyboard and mouse message in the second operation mode so as to operate the identification end locally.

4. The method according to claim 3, wherein: The identification end includes a keyboard and mouse component, and the keyboard and mouse component includes a keyboard and / or a mouse. The sending of a keyboard and mouse message to the target security inspection end based on a preset operation instruction includes: In the first operation mode, the automatically generated keyboard and mouse message and / or the keyboard and mouse message manually input by the user operating the keyboard and mouse component are sent to the target security inspection terminal.

5. The method according to claim 4, wherein: The method further comprises: In the second operation mode, the identification terminal is operated locally in response to the automatically generated keyboard and mouse information and / or the manually input keyboard and mouse message.

6. The method according to any one of claims 1 to 5, wherein: The identification end includes an identification computer and a keyboard and mouse simulation component, the target security inspection end includes a target security inspection computer connected to the security inspection machine, and the method further includes: Providing the keyboard and mouse simulation component to connect the identification computer and the target security inspection computer; The sending of the keyboard and mouse message to the target security inspection terminal includes: After identifying a suspicious object, the identification computer sends the keyboard and mouse message to the target security inspection computer through the keyboard and mouse simulation component, and the keyboard and mouse simulation component is used to simulate the keyboard message and / or mouse message of the target security inspection computer.

7. The method according to claim 6, wherein: Before performing suspicious object identification on the video data from the target security inspection end, the method further includes: Collecting the video signal of the target security inspection computer to obtain the video data; Wherein, the identification computer is connected to a display, and the identification of suspicious objects includes: Performing suspicious object recognition on the image frame of the object to be detected in the video data; The suspicious object identification conclusion and the suspicious object label are output to the display together with the video data for superimposed display.

8. The method according to claim 7, wherein: The superimposed display includes: The video parameters are superimposed and displayed according to the video parameters at the time of acquisition, wherein the video parameters include the video resolution and frame rate of the video signal.

9. A security inspection machine operating system, comprising: an identification computer connected to N security inspection computers, and configured to execute the method according to any one of claims 1 to 8, wherein N is an integer greater than or equal to 1; An external device is connected to the identification computer, wherein the external device includes at least one of a keyboard, a mouse or a display, wherein at least one target security inspection computer among the N security inspection computers is configured to share the external device with the identification computer.

10. A security inspection machine operating device, used for identification terminal, comprising: An object recognition module, used to identify suspicious objects from video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection terminal, the target security inspection terminal being any one of N security inspection terminals, where N is an integer greater than or equal to 1; The keyboard and mouse message module is used to send keyboard and mouse messages to the target security inspection end based on preset operation instructions after identifying a suspicious object, wherein the keyboard and mouse messages include keyboard messages and / or mouse messages, and the target security inspection end is configured to execute the preset operation instructions in response to the keyboard and mouse messages.

11. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images