Security inspection machine operation method, system, device, equipment, medium and program product

Abstract

The invention provides a security inspection machine operation method for an identification end, and relates to the field of security inspection. The method comprises the following steps: performing suspicious object identification on video data from a target security check end; after the suspicious object is recognized, a keyboard and mouse message is sent to the target security check terminal based on a preset operation instruction, and the target security check terminal is configured to respond to the keyboard and mouse message to execute the preset operation instruction. According to the embodiment of the invention, after the suspicious object is identified, the target security inspection machine end of a third-party manufacturer can receive and identify the preset operation instruction in time in the form of the keyboard and mouse message under the condition that the business system of the target security inspection end is not changed, so that the target security inspection end executes the preset operation instruction to realize specific operation. The invention also provides a security inspection machine operating system, a security inspection machine device, security inspection equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of security inspection, and more specifically, to a security inspection machine operation method, system, device, equipment, medium and program product. Background Art

[0002] In the security inspection industry, the data formats or software business systems of security inspection machines from different manufacturers are incompatible. If you want to provide artificial intelligence (AI) recognition services for third-party security inspection machines, it may not be possible to implement it directly on the security inspection machine. You can provide independent AI recognition devices for each third-party security inspection machine to implement AI recognition functions.

[0003] Because the data formats or software business systems of different manufacturers are incompatible, and the communication interfaces and protocols between the security inspection machines of third-party manufacturers and AI recognition devices may be different, or the third-party manufacturers encrypt the communication interfaces, the AI ​​recognition devices cannot directly operate one or more security inspection machines of third-party manufacturers through network protocols for timely processing after identifying suspicious objects. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a security inspection machine operation method, system, device, equipment, medium and program product.

[0005] One aspect of an embodiment of the present disclosure provides a security inspection machine operation method for an identification terminal, comprising: identifying suspicious objects on video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by a security inspection machine of the target security inspection terminal, the target security inspection terminal being any one of N security inspection terminals, and N being an integer greater than or equal to 1; after identifying a suspicious object, sending a keyboard and mouse message to the target security inspection terminal based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection terminal is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0006] According to an embodiment of the present disclosure, before sending a keyboard and mouse message to the target security inspection end based on a preset operation instruction, the method also includes: monitoring the keyboard and mouse message generated locally by the identification end; after monitoring the keyboard and mouse message, obtaining the current operation mode of the identification end to determine whether to send the keyboard and mouse message.

[0007] According to an embodiment of the present disclosure, the current operation mode includes a first operation mode or a second operation mode. Before determining the current operation mode of the identification end, the method further includes: switching to the first operation mode in response to a first mode instruction, wherein the identification end is configured to allow the sending of the keyboard and mouse messages in the first operation mode; and / or switching to the second operation mode in response to a second mode instruction, wherein the identification end is configured to prohibit the sending of the keyboard and mouse messages in the second operation mode, so as to operate the identification end locally.

[0008] According to an embodiment of the present disclosure, the identification end includes a keyboard and mouse component, and the keyboard and mouse component includes a keyboard and / or a mouse. The sending of keyboard and mouse messages to the target security inspection end based on preset operation instructions includes: in the first operation mode, sending the automatically generated keyboard and mouse messages to the target security inspection end, and / or the keyboard and mouse messages manually input by the user operating the keyboard and mouse component.

[0009] According to an embodiment of the present disclosure, the method further includes: in the second operation mode, in response to the automatically generated keyboard and mouse information and / or the manually input keyboard and mouse message, operating the identification end locally.

[0010] According to an embodiment of the present disclosure, the identification end includes an identification computer and a keyboard and mouse simulation component, the target security inspection end includes a target security inspection computer connected to the security inspection machine, and the method further includes: providing the keyboard and mouse simulation component to connect the identification computer and the target security inspection computer; wherein, sending a keyboard and mouse message to the target security inspection end includes: after the identification computer identifies a suspicious object, the keyboard and mouse simulation component sends the keyboard and mouse message to the target security inspection computer through the keyboard and mouse simulation component, and the keyboard and mouse simulation component is used to simulate the keyboard message and / or mouse message of the target security inspection computer.

[0011] According to an embodiment of the present disclosure, before performing suspicious object identification on the video data from the target security inspection end, the method further includes: acquiring the video signal of the target security inspection computer to obtain the video data; wherein the identification computer is connected to a display, and the performing suspicious object identification includes: performing suspicious object identification on the image frame of the object to be inspected in the video data; and outputting the suspicious object identification conclusion and the suspicious object label with the video data to the display for superimposed display.

[0012] According to an embodiment of the present disclosure, the superimposed display includes: superimposed display according to video parameters during acquisition, wherein the video parameters include a video resolution and a frame rate of the video signal.

[0013] Another aspect of the present disclosure provides a security inspection machine operating system, including:

[0014] an identification computer connected to N security inspection computers, and configured to execute the method described in any one of the above items, where N is an integer greater than or equal to 1;

[0015] An external device is connected to the identification computer, wherein the external device includes at least one of a keyboard, a mouse or the display, wherein at least one target security inspection computer among the N security inspection computers is configured to share the external device with the identification computer.

[0016] Another aspect of the disclosed embodiment provides a security inspection machine operating device for use at an identification terminal, including:

[0017] An object recognition module, used to identify suspicious objects from video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection terminal, the target security inspection terminal being any one of N security inspection terminals, where N is an integer greater than or equal to 1;

[0018] The keyboard and mouse message module is used to send keyboard and mouse messages to the target security inspection end based on preset operation instructions after identifying a suspicious object, wherein the keyboard and mouse messages include keyboard messages and / or mouse messages, and the target security inspection end is configured to execute the preset operation instructions in response to the keyboard and mouse messages.

[0019] The security inspection machine operating device includes modules respectively used to execute each step of any one of the methods described above.

[0020] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method as described above.

[0021] Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.

[0022] Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0023] One or more of the above-mentioned embodiments have the following beneficial effects: different from the method of directly operating the security inspection machines of various third-party manufacturers through network protocols, it can overcome obstacles such as different communication interfaces and protocols, incompatible data formats or encryption of communication interfaces by third-party manufacturers. After identifying suspicious objects in the video data of the target security inspection end, without changing the business system of the target security inspection end, the target security inspection machine of the third-party manufacturer can promptly receive and identify the preset operation instructions in the form of keyboard and mouse messages, so that the target security inspection end can execute the preset operation instructions to achieve specific operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0025] Figure 1 The application scenario diagram of the security inspection machine operation method according to the embodiment of the present disclosure is schematically shown;

[0026] Figure 2 A flowchart schematically shows a method for operating a security inspection machine according to an embodiment of the present disclosure;

[0027] Figure 3a Schematically shows a flow chart for determining a current operation mode according to an embodiment of the present disclosure;

[0028] Figure 3b The schematic diagram of the architecture of using a keyboard and mouse switch according to an embodiment of the present disclosure is schematically shown.

[0029] Figure 4 The flowchart of sending keyboard and mouse messages according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 5 A schematic diagram showing an application scenario of a method for operating a security inspection machine according to another embodiment of the present disclosure is shown;

[0031] Figure 6 Schematically shows a flowchart of overlay display according to an embodiment of the present disclosure;

[0032] Figure 7 A schematic diagram schematically shows a separate display of video data according to an embodiment of the present disclosure;

[0033] Figure 8 A schematic diagram schematically shows a video data overlay display according to an embodiment of the present disclosure;

[0034] Fig. 9 A schematic diagram of a structure of a security inspection machine operating device according to an embodiment of the present disclosure is shown; and

[0035] Fig.10A block diagram of an electronic device suitable for implementing a security inspection machine operating method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0036] In order to facilitate understanding of the technical solutions of the embodiments of the present application, some technical terms involved in the present application are first introduced.

[0037] Recognition end: includes hardware and software to implement AI recognition functions.

[0038] Security inspection end: includes hardware and software for implementing security inspection functions.

[0039] Security inspection machine: belongs to the security inspection end, including scanning equipment and security inspection computer. For example, the scanning equipment may include X-ray machine, conveyor belt and security inspection channel.

[0040] Security inspection computer: includes a host computer, which is connected to the scanning device to receive scanning data to form a video signal, or issue scanning device control instructions.

[0041] Security inspection machine operating system: belongs to the identification end, including the identification computer, keyboard and mouse simulation components and their external devices and other hardware and software systems.

[0042] Identify the computer: including the computer host. The computer host refers to the main body of the computer excluding peripherals such as input and output devices.

[0043] Keyboard messages: include key messages and character messages. The underlying operating system (such as Windows or Linux) can generate keyboard messages in response to application instructions (such as a virtual keyboard, or calling the keyboard interface to directly input instructions), keyboard input signals, touch input, voice control, etc.

[0044] Mouse message: Mouse movement, click or scroll wheel actions will generate mouse messages, which are obtained by the data sent by the mouse's internal MCU to the host. In addition, mouse actions can also be operated through keyboard, touch input, voice control, etc. to generate mouse messages.

[0045] Preset operation instructions: instructions that can affect the action or status of the security inspection machine, such as the emergency stop instruction, start instruction, alarm instruction, conveyor belt forward instruction, acceleration and deceleration instruction of the scanning equipment, etc.

[0046] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0047] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0048] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0049] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0050] If you want to provide AI intelligent recognition services for security inspection machines from different manufacturers, you can perform intelligent analysis on the video stream connected to the security inspection machine, structure the video packages in the video stream, perform AI intelligent recognition in real time, and provide AI conclusions for the image judges. The above video access method can provide intelligent image review services for security inspection machines from different manufacturers, with good compatibility, and can avoid the technical barriers caused by the compatibility issues of the original data formats of different manufacturers. However, there is still the problem that after the AI ​​recognition device identifies a suspicious object, it cannot directly operate the security inspection machines of various third-party manufacturers through the network protocol for timely processing.

[0051] The embodiments of the present disclosure provide a security inspection machine operation method, system, device, equipment, medium and program product, which are different from the method of directly operating the security inspection machines of third-party manufacturers through network protocols. They can overcome obstacles such as different communication interfaces and protocols, incompatible data formats or encryption of communication interfaces by third-party manufacturers. After suspicious objects are identified in the video data of the target security inspection end, the target security inspection machine of the third-party manufacturer can promptly receive and identify preset operation instructions in the form of keyboard and mouse messages, so that the target security inspection end can execute the preset operation instructions to achieve specific operations.

[0052] Figure 1 The application scenario diagram of the security inspection machine operation method according to the embodiment of the present disclosure is schematically shown. However, the present disclosure is not limited to Figure 1 The application scenario and architecture shown.

[0053] like Figure 1 The application scenario 100 shown may include an identification terminal and a security inspection terminal. The identification terminal may include a security inspection machine operating system 110. The security inspection terminals may include security inspection terminals 1-121, security inspection terminals 2-122, ... security inspection terminals N-12N. The security inspection machine operating system 110 may include an identification computer 111 and an external device 112. The security inspection machine operating system 110 may be connected to at least one of the security inspection terminals 1-N by communication, for example, by directly connecting using the identification computer 111, or by setting other hardware between the identification computer 111 and the security inspection terminals 1-N to achieve the connection.

[0054] In some embodiments, each of the security inspection terminals 1 to N includes a security inspection computer. The identification computer 111 can be connected to N security inspection computers to execute the security inspection machine operation method provided in one or more embodiments of the present disclosure, where N is an integer greater than or equal to 1. The external device 112 is connected to the identification computer 111, and the external device 112 includes at least one of a keyboard, a mouse, or a display, wherein at least one target security inspection computer among the N security inspection computers is configured to share an external device with the identification computer.

[0055] In one or more embodiments of the present disclosure, in addition to the security inspection computer and the identification computer set up locally at the security inspection site, at least one of the security inspection computer and the identification computer may also be replaced by a cloud host set up remotely based on cloud computing technology.

[0056] According to the embodiments of the present disclosure, the security inspection computer and the identification computer share external devices, which can avoid the confusion caused by multiple sets of external devices, save external device costs and equipment layout space, and improve the operational convenience of multiple security inspection machines.

[0057] It is understandable that the present disclosure is not intended to be limited to relying on the target security inspection computer and the identification computer to share external devices to overcome the defects in the related art. For example, each security inspection computer can have a matching external device, or at least two security inspection computers can share a set of external devices, which can be set according to actual conditions.

[0058] The following will be based on Figure 1 The scene described by Figure 2 to Figure 8 The operating method of the security inspection machine according to the embodiment of the present disclosure is described in detail.

[0059] Figure 2 The flowchart of the security inspection machine operation method according to the embodiment of the present disclosure is schematically shown.

[0060] like Figure 2 As shown, the security inspection machine operation method of this embodiment is used for the identification end, including operations S210 to S230.

[0061] In operation S210, suspicious object identification is performed on video data from a target security inspection end, where the video data is obtained by scanning an object to be inspected by a security inspection machine at the target security inspection end. The target security inspection end is any one of N security inspection ends, where N is an integer greater than or equal to 1.

[0062] Exemplarily, the objects to be inspected may include luggage, parcels, packaging bags and other contents and the items therein, and may also include persons and carry-on items. The scanning method of the security inspection machine may include one or more of X-ray, millimeter wave or visible light. The recognition end may obtain the video data of the target security inspection end in real time or asynchronously through wired or wireless means, and similarly, may also perform real-time or asynchronous suspicious object identification. Suspicious objects include prohibited items such as weapons, ammunition, highly toxic substances or flammable and explosive substances, and may also include other objects specified by the management of public places.

[0063] In some embodiments, the recognition process may include: obtaining an X-ray image of the package to be detected. Performing object detection on the X-ray image based on the pre-stored template in the contour matching template library to obtain a suspicious object recognition conclusion. The recognition process may also include: inputting the X-ray image into an artificial intelligence model, such as a classification model implemented based on a machine learning algorithm, and obtaining a suspicious object recognition conclusion. The recognition process may also include: installing a camera above the entrance of the X-ray machine and installing identification strips on both sides of the conveyor belt. Using a deep focused convolutional residual network to perform object detection on visible light images and X-ray scan images.

[0064] In operation S220, after a suspicious object is identified, a keyboard and mouse message is sent to a target security inspection terminal based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection terminal is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0065] In some embodiments, the recognition end may simultaneously recognize video data from one or more target security inspection ends, and send keyboard and mouse messages to the corresponding target security inspection ends after a suspicious object is recognized.

[0066] For example, there may be multiple preset operation instructions, for example, the corresponding preset operation instructions are determined according to the category of the suspicious object, and each preset operation instruction has a corresponding keyboard and mouse message. The identification end may send the keyboard and mouse message to the target security inspection end by wired connection or wireless connection.

[0067] It can be understood that after the target security inspection end receives the keyboard and mouse message, the security inspection machine believes that it has received the keyboard and mouse message generated by an external device or locally, so it can recognize the keyboard and mouse message and execute the corresponding preset operation instructions.

[0068] In some embodiments, the recognition computer may generate keyboard and mouse messages locally, or may accept user input, such as keyboard and mouse input, or voice control input, touch input, or gesture input to generate keyboard and mouse messages.

[0069] According to the embodiments of the present disclosure, unlike the method of directly operating the security inspection machines of various third-party manufacturers through network protocols, it is possible to overcome obstacles such as different communication interfaces and protocols, incompatible data formats, or encryption of communication interfaces by third-party manufacturers. After a suspicious object is identified in the video data of the target security inspection end, the target security inspection machine of the third-party manufacturer can promptly receive and identify the preset operation instructions in the form of keyboard and mouse messages, thereby allowing the target security inspection end to execute the preset operation instructions to achieve a specific operation.

[0070] Figure 3a The flowchart for determining the current operation mode according to an embodiment of the present disclosure is schematically shown.

[0071] Before sending keyboard and mouse messages to the target security inspection terminal based on the preset operation instructions, Figure 3a As shown, this embodiment includes operations S310 to S320.

[0072] In operation S310, keyboard and mouse messages generated locally by the recognition terminal are monitored.

[0073] For example, deploy an application to monitor the keyboard and mouse events of the Windows system, and implement monitoring and identification of the underlying keyboard and mouse messages.

[0074] In operation S320, after the keyboard and mouse message is monitored, the current operation mode of the recognition end is obtained to determine whether to send the keyboard and mouse message.

[0075] Exemplarily, the operation mode is used to set whether to execute the forwarding action of the keyboard and mouse message.

[0076] According to an embodiment of the present disclosure, the sending of the keyboard and mouse message is further confirmed through the current operation mode to avoid erroneous operation on the target security inspection end when the keyboard and mouse message does not need to be sent.

[0077] In some embodiments, the current operation mode includes the first operation mode or the second operation mode, and before determining the current operation mode of the identification terminal, it further includes: responding to the first mode instruction to switch to the first operation mode, wherein the identification terminal is configured to allow the sending of keyboard and mouse messages in the first operation mode. And / or responding to the second mode instruction to switch to the second operation mode, wherein the identification terminal is configured to prohibit the sending of keyboard and mouse messages in the second operation mode, so as to operate the identification terminal locally.

[0078] Exemplarily, the forwarding action is executed in the first operation mode, and the forwarding action is stopped in the second operation mode. The first mode instruction or the second mode instruction includes, for example, one or more preset shortcut key instructions, voice instructions, or gesture instructions. The recognition end may include software and hardware such as a recognition computer and its application programs.

[0079] In some embodiments, when the security inspection computer needs to be operated, the pre-set shortcut keys are used to switch, and the recognition computer completely forwards the collected mouse messages and keyboard messages to the security inspection computer. When the recognition computer is operated, it is only necessary to switch according to the pre-set shortcut keys, and all messages are not forwarded or distributed to the security inspection computer.

[0080] In the related art, the security inspection computer and the identification computer are two different computers, and each needs to be provided with a set of mouse and keyboard sets. One set of keyboard and mouse is needed to operate the security inspection machine, and another set of keyboard and mouse is needed to operate the identification computer, which brings great inconvenience to on-site personnel. It is easy to confuse the two sets of mouse and keyboard on the same operating table.

[0081] Figure 3b The schematic diagram of the architecture of using a keyboard and mouse switch according to an embodiment of the present disclosure is schematically shown.

[0082] In some embodiments, reference Figure 3b , an independent keyboard and mouse switch can be set up to realize that the security inspection computers 1 to N and the identification computer share a set of mouse and keyboard, avoiding the confusion of multiple sets of mouse and keyboard on one operating table. The keyboard and mouse switch can synchronously transmit the keyboard and mouse signals to each controlled computer by setting different shortcut keys, and can realize the operation of a single or multiple controlled computers to meet the needs of keyboard and mouse switching.

[0083] However, the keyboard and mouse switch also has certain defects. Each computer is independent of each other and passively receives the key and mouse signals from the keyboard and mouse switch. If the recognition computer identifies high-risk goods, it needs to actively provide emergency stop service or personalized notification service, and the keyboard and mouse switch cannot meet this business demand.

[0084] According to the embodiment of the present disclosure, in the first operation mode, the keyboard and mouse messages are forwarded to the target security inspection terminal to implement the operation on the target security inspection terminal. In the second operation mode, the keyboard and mouse messages are not forwarded to the target security inspection terminal to implement the local operation. Therefore, the target security inspection terminal and the identification terminal can share a set of mouse and keyboard, and realize that the target security inspection terminal of the third-party manufacturer can receive and identify the preset operation instructions in time through the form of keyboard and mouse messages, so that the target security inspection terminal executes the preset operation instructions to implement specific operations.

[0085] In some embodiments, the identification end includes a keyboard and mouse component, which includes a keyboard and / or a mouse. In operation S220, sending a keyboard and mouse message to the target security inspection end based on a preset operation instruction includes: in a first operation mode, sending an automatically generated keyboard and mouse message to the target security inspection end, and / or a keyboard and mouse message manually input by a user operating the keyboard and mouse component.

[0086] For example, the keyboard may include a general keyboard with an A-Z letter area, a 0-9 number area and other symbol areas, or may include a customized keyboard adapted to the functions of the identification end or the target security inspection machine end. Similarly, the mouse may also be general or customized.

[0087] According to the embodiments of the present disclosure, in the security inspection scenario, the time for each package to be inspected is roughly the time between the input port and the output port of the conveyor belt, so the time is limited. After identifying a suspicious object, a keyboard and mouse message is automatically generated to operate the target security inspection terminal, which can have a high timeliness. In addition, through the keyboard and mouse components, users can set and operate the target security inspection terminal according to their own preferences, providing more operation methods and improving convenience.

[0088] In some embodiments, it also includes: in the second operation mode, in response to automatically generated keyboard and mouse information and / or manually input keyboard and mouse messages, operating the identification end locally.

[0089] According to an embodiment of the present disclosure, when operating the identification computer, switching is implemented according to the pre-set second mode instructions, and all messages are not forwarded or distributed to the security inspection computer, thereby enabling automated operation and / or manual operation, thereby improving the operational convenience of the identification terminal.

[0090] Figure 4 The flowchart of sending keyboard and mouse messages according to an embodiment of the present disclosure is schematically shown. Figure 5An application scenario diagram of a security inspection machine operating method according to another embodiment of the present disclosure is schematically shown.

[0091] like Figure 4 As shown, the sending of the keyboard and mouse message in this embodiment includes operations S410 to S420. Operation S420 is one embodiment of operation S220. The identification end includes an identification computer and a keyboard and mouse simulation component.

[0092] In operation S410, a keyboard and mouse simulation component is provided to connect the identification computer and the target security inspection computer.

[0093] Exemplarily, the keyboard and mouse simulation component may include keyboard and mouse simulation hardware and / or keyboard and mouse simulation software. In some embodiments, the hardware may be a wired connector. In other embodiments, the hardware may be a wireless connector, for example, a converter module is inserted into the identification computer and the target security inspection computer respectively, and the two converter modules can be wirelessly connected. In other embodiments, the software can virtualize one or more serial ports in the computer, for example, the identification computer and the target security inspection computer respectively virtualize serial ports that can directly communicate with each other, or set up a serial port server to realize the forwarding of keyboard and mouse messages from the identification computer to the target security inspection computer.

[0094] In operation S420, after identifying a suspicious object, the identification computer sends a keyboard and mouse message to the target security inspection computer through the keyboard and mouse simulation component, where the keyboard and mouse simulation component is used to simulate keyboard messages and / or mouse messages of the target security inspection computer.

[0095] For example, the keyboard and mouse simulation component can provide a virtual serial port to convert the data generated by the computer to realize the function of simulating keyboard messages and / or mouse messages, such as using WG Wiegand to USB Keyboard to simulate the keyboard and virtual serial port. Figure 5 As shown, in the security inspection computer 1 of the security inspection end 1, the virtual serial port is recognized as a real port by the operating system and the serial application, and the input of the virtual serial port can be simulated as keyboard input, thereby generating recognizable keyboard messages in the security inspection computer, thereby realizing communication between the identification end and the target security inspection end.

[0096] According to the embodiments of the present disclosure, the security inspection computer and the identification computer are two different computers, and the keyboard and mouse simulation components realize keyboard and mouse message routing and forwarding. By pre-setting mode instructions (such as shortcut keys), a common set of keyboard and mouse components is realized, which brings convenience to the user's operation.

[0097] Figure 6 The flowchart of overlay display according to an embodiment of the present disclosure is schematically shown. Figure 7 The figure schematically shows a schematic diagram of separate display of video data according to an embodiment of the present disclosure. Figure 8The diagram schematically shows the video data overlay display according to an embodiment of the present disclosure.

[0098] like Figure 6 As shown, the superimposed display of this embodiment includes operations S610 to S630. Among them, it is recognized that the computer is connected to a display, and operations S620 and S630 are embodiments of operation S210.

[0099] In operation S610, a video signal of a target security inspection computer is collected to obtain video data.

[0100] In operation S620, suspicious object recognition is performed on the image frame of the object to be detected in the video data.

[0101] Exemplarily, the image frame is an image containing an object to be inspected, which can be subjected to AI recognition.

[0102] In operation S630, the suspicious object recognition conclusion, the suspicious object annotation and the video data are output to a display for superimposed display.

[0103] For example, the suspicious object recognition conclusion includes a suspicious object existence conclusion or a normal conclusion, and the suspicious object annotation may include suspicious object category information and a standard frame for selecting the suspicious object in the image.

[0104] In some embodiments, the display is overlaid based on video parameters during acquisition, where the video parameters include video resolution and frame rate of the video signal.

[0105] Reference Figure 5 The recognition computer 111 collects the video signal (VGA port, DVI port or HDMI port, etc.) of the security inspection computer 1, collects the video data in the original video resolution and frame rate, performs video object tracking analysis, package segmentation and other processing on each frame of the collected video data, and performs AT intelligent recognition at the same time, marks the suspect conclusion of AT recognition and the video layer of the original video frame as a suspect, and displays the superimposed video picture according to the original collected video resolution and frame rate and outputs it to the display.

[0106] Users can check key items according to their own preferences, such as explosives, guns, controlled knives, etc. If the above suspects are found during the machine security check, a specified keyboard message can be sent to the security computer 1 to stop the tape or give a warning prompt. Figure 5 As shown, the keyboard and mouse are connected to the identification computer 111, and the keyboard and mouse messages are intercepted by the identification computer 111, and distributed and processed to the security inspection computer 1 according to the business demand function. The keyboard and mouse message sending can be customized to realize automatic emergency stop or warning prompts, etc., providing a method for interaction between different computers.

[0107] For example, the display screen of the security inspection computer 1 connected to the display is as follows: Figure 7 , the recognition computer 111 collects the video stream of the security inspection computer 1, and then superimposes the display result on the display, and the display screen is as follows Figure 8 , which is Figure 7 On this basis, annotation boxes and conclusions are superimposed.

[0108] According to the embodiment of the present disclosure, the security inspection computer 1 and the identification computer 111 can share the same display screen and mouse and keyboard, thereby avoiding confusion and interference caused by multiple display screens to the reviewers and reducing the cost of deploying multiple display screens.

[0109] According to the embodiments of the present disclosure, displaying according to the originally acquired video parameters can ensure the overlay display effect of the video data on the display, avoiding distortion such as an overly large annotation box, elongated wrapped image, or missing part of the image.

[0110] Based on the above security inspection machine operation method, the present disclosure also provides a security inspection machine operation device. Fig. 9 The device is described in detail.

[0111] Fig. 9 The structural block diagram of the security inspection machine operating device according to an embodiment of the present disclosure is schematically shown.

[0112] like Fig. 9 As shown, the security inspection machine operating device 900 of this embodiment includes an object recognition module 910 and a keyboard and mouse message module 920.

[0113] The object recognition module 910 can perform operation S210 to identify suspicious objects in video data from a target security inspection end, where the video data is obtained by scanning the object to be inspected by a security inspection machine at the target security inspection end. The target security inspection end is any one of N security inspection ends, where N is an integer greater than or equal to 1.

[0114] The keyboard and mouse message module 920 can execute operation S220, which is used to send a keyboard and mouse message to the target security inspection end based on a preset operation instruction after identifying a suspicious object, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message.

[0115] In some embodiments, the security inspection machine operating device 900 may further include a mode determination module, which may execute operations S310 to S320, which will not be described in detail here.

[0116] In some embodiments, the security inspection machine operating device 900 may also include a mode switching module for switching to a first operating mode in response to a first mode instruction, wherein the identification end is configured to allow the sending of keyboard and mouse messages in the first operating mode; and / or switching to a second operating mode in response to a second mode instruction, wherein the identification end is configured to prohibit the sending of keyboard and mouse messages in the second operating mode, so as to operate the identification end locally.

[0117] In some embodiments, the keyboard and mouse message module 920 is also used to send automatically generated keyboard and mouse messages and / or keyboard and mouse messages manually input by the user operating the keyboard and mouse components to the target security inspection terminal in the first operation mode. Or in the second operation mode, in response to the automatically generated keyboard and mouse information and / or the manually input keyboard and mouse messages, operate the identification terminal locally.

[0118] In some embodiments, a keyboard and mouse simulation component is provided to connect the identification computer and the target security inspection computer. The keyboard and mouse message module 920 is also used to enable the identification computer to send keyboard and mouse messages to the target security inspection computer through the keyboard and mouse simulation component after identifying a suspicious object. The keyboard and mouse simulation component is used to simulate the keyboard messages and / or mouse messages of the target security inspection computer.

[0119] In some embodiments, the security inspection machine operating device 900 may further include a video acquisition module for acquiring a video signal of a target security inspection computer to obtain video data (operation S610). The object recognition module 910 is also used to perform operations S620 to S630, which will not be described in detail here.

[0120] In some embodiments, the object recognition module 910 is further used for overlay display according to video parameters during acquisition, where the video parameters include video resolution and frame rate of the video signal.

[0121] It should be noted that the security inspection machine operating device 900 includes two parts for performing the above Figure 2 to Figure 7 The modules of each step of any embodiment described. The implementation methods, technical problems solved, functions realized, and technical effects achieved of each module / unit / subunit, etc. in the device part embodiment are respectively the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0122] According to an embodiment of the present disclosure, any multiple modules in the object recognition module 910 and the keyboard and mouse message module 920 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0123] According to an embodiment of the present disclosure, at least one of the object recognition module 910 and the keyboard and mouse message module 920 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the object recognition module 910 and the keyboard and mouse message module 920 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0124] Fig.10 A block diagram of an electronic device suitable for implementing a security inspection machine operating method according to an embodiment of the present disclosure is schematically shown.

[0125] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0126] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0127] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc. An output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. A storage portion 1008 including a hard disk, etc. And a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that the computer program read therefrom is installed into the storage portion 1008 as needed.

[0128] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments. It may also exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0129] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0130] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0131] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0132] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0133] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0134] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0135] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0137] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for operating a security inspection machine, used for an identification terminal, comprising: Perform suspicious object identification on video data from a target security inspection end, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection end, the target security inspection end being any one of N security inspection ends, where N is an integer greater than or equal to 1; After identifying a suspicious object, a keyboard and mouse message is sent to the target security inspection end based on a preset operation instruction, wherein the keyboard and mouse message includes a keyboard message and / or a mouse message, and the target security inspection end is configured to execute the preset operation instruction in response to the keyboard and mouse message.

2. The method according to claim 1, wherein: Before sending a keyboard and mouse message to the target security inspection terminal based on a preset operation instruction, the method further includes: Monitor the keyboard and mouse messages generated locally by the recognition terminal; After monitoring the keyboard and mouse message, the current operation mode of the identification terminal is obtained to determine whether to send the keyboard and mouse message.

3. The method according to claim 2, wherein: The current operation mode includes a first operation mode or a second operation mode. Before determining the current operation mode of the identification terminal, the method further includes: In response to a first mode instruction, switching to the first operation mode, wherein the identification end is configured to allow sending the keyboard and mouse message in the first operation mode; and / or In response to the second mode instruction, the second operation mode is switched, wherein the identification end is configured to prohibit sending the keyboard and mouse message in the second operation mode so as to operate the identification end locally.

4. The method according to claim 3, wherein: The identification end includes a keyboard and mouse component, and the keyboard and mouse component includes a keyboard and / or a mouse. The sending of a keyboard and mouse message to the target security inspection end based on a preset operation instruction includes: In the first operation mode, the automatically generated keyboard and mouse message and / or the keyboard and mouse message manually input by the user operating the keyboard and mouse component are sent to the target security inspection terminal.

5. The method according to claim 4, wherein: The method further comprises: In the second operation mode, the identification terminal is operated locally in response to the automatically generated keyboard and mouse information and / or the manually input keyboard and mouse message.

6. The method according to any one of claims 1 to 5, wherein: The identification end includes an identification computer and a keyboard and mouse simulation component, the target security inspection end includes a target security inspection computer connected to the security inspection machine, and the method further includes: Providing the keyboard and mouse simulation component to connect the identification computer and the target security inspection computer; The sending of the keyboard and mouse message to the target security inspection terminal includes: After identifying a suspicious object, the identification computer sends the keyboard and mouse message to the target security inspection computer through the keyboard and mouse simulation component, and the keyboard and mouse simulation component is used to simulate the keyboard message and / or mouse message of the target security inspection computer.

7. The method according to claim 6, wherein: Before performing suspicious object identification on the video data from the target security inspection end, the method further includes: Collecting the video signal of the target security inspection computer to obtain the video data; Wherein, the identification computer is connected to a display, and the identification of suspicious objects includes: Performing suspicious object recognition on the image frame of the object to be detected in the video data; The suspicious object identification conclusion and the suspicious object label are output to the display together with the video data for superimposed display.

8. The method according to claim 7, wherein: The superimposed display includes: The video parameters are superimposed and displayed according to the video parameters at the time of acquisition, wherein the video parameters include the video resolution and frame rate of the video signal.

9. A security inspection machine operating system, comprising: an identification computer connected to N security inspection computers, and configured to execute the method according to any one of claims 1 to 8, wherein N is an integer greater than or equal to 1; An external device is connected to the identification computer, wherein the external device includes at least one of a keyboard, a mouse or a display, wherein at least one target security inspection computer among the N security inspection computers is configured to share the external device with the identification computer.

10. A security inspection machine operating device, used for identification terminal, comprising: An object recognition module, used to identify suspicious objects from video data from a target security inspection terminal, the video data being obtained by scanning an object to be inspected by a security inspection machine at the target security inspection terminal, the target security inspection terminal being any one of N security inspection terminals, where N is an integer greater than or equal to 1; The keyboard and mouse message module is used to send keyboard and mouse messages to the target security inspection end based on preset operation instructions after identifying a suspicious object, wherein the keyboard and mouse messages include keyboard messages and / or mouse messages, and the target security inspection end is configured to execute the preset operation instructions in response to the keyboard and mouse messages.

11. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images