Methods, systems and main control equipment for preventing human errors in the main control room of a nuclear power plant

Abstract

The present invention discloses a method, system and main control equipment for preventing human errors in the main control room of a nuclear power plant. The method comprises: acquiring an image sequence composed of images of an operator workstation display interface in real time, and detecting and identifying images in the image sequence to obtain a manual input control command for the operator to operate the unit; acquiring the vote-calling voice information of the operator workstation in real time, and identifying the vote-calling voice information to obtain a notification control command for the operator to notify the guardian; comparing the manual input control command with the notification control command, and issuing an early warning when the two are inconsistent. The present invention obtains the input control command and the notification control command of the operator workstation in real time, and when the two are found to be inconsistent, an early warning is issued in time to remind the operator, thereby preventing the occurrence of nuclear power plant operation safety accidents caused by human factors of the operator, and improving the safety of nuclear power plant control.

Description

Technical Field

[0001] The invention belongs to the technical field of nuclear power, and in particular relates to a method, system and main control equipment for preventing human errors in a main control room of a nuclear power plant. Background Art

[0002] In the digital control room of a nuclear power plant, operators mainly use monitors, mice and keyboards as media to complete process control and status monitoring of the nuclear power plant.

[0003] According to the requirements of the operator code of conduct of nuclear power plants, important safety-related operations should be performed in accordance with the monitoring system. That is, the operator should loudly inform the monitor of the equipment code of the controlled object and the upcoming control instructions. After the monitor confirms that they are correct, he will give permission to the operator. The operator inputs and issues control commands through the mouse, keyboard, etc., and the control commands are displayed on the monitor at the same time, so as to prevent human errors caused by erroneous operations.

[0004] However, in the actual operation of the unit, the operator has a heavy workload. Especially when the unit status changes, the operator often needs to issue a large number of control commands continuously. In high-intensity operations, operators and supervisors are prone to make mistakes due to their inability to concentrate for a long time. Once the control equipment or instructions reported by the operator are inconsistent with the actual input, it may cause serious consequences to the nuclear power plant and affect the safe and stable operation of the nuclear power plant. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method, system and main control equipment for preventing human errors in the main control room of a nuclear power plant in view of the above-mentioned deficiencies in the prior art, so as to solve the problem that the operator's manually input control commands are inconsistent with the vote-calling control commands, which affects the safety of nuclear power plant operation, so as to achieve safe operation of the nuclear power plant.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for preventing human errors in a main control room of a nuclear power plant, the method comprising:

[0008] Acquire in real time an image sequence composed of images of the operator workstation display interface, and detect and identify images in the image sequence to obtain a manual input control command for the operator to operate the unit;

[0009] Acquire the voice information of the vote counting of the operator workstation in real time, and recognize the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian;

[0010] The manually input control command and the notification control command are compared, and an early warning is issued when the two are inconsistent.

[0011] Preferably, the real-time acquisition of an image sequence composed of operator workstation display interface images, and the detection and recognition of images in the image sequence to obtain a manual input control command for the operator to operate the unit, specifically includes:

[0012] receiving in real time a video stream of the operator workstation display interface acquired by a camera, and acquiring images at each moment within a preset time length from the video stream in chronological order to form an image sequence;

[0013] The image sequence is detected using a pre-trained convolutional neural network-based target detection algorithm to obtain the image and location of the manual input control command of the operator operating the unit;

[0014] A text recognition algorithm based on a convolutional recurrent neural network is used for the location of the image obtained by detection to obtain the text results of the manual input control commands of the operator operating the unit.

[0015] Preferably, the real-time receiving of the video stream of the display interface when the operator operates the unit collected by the camera, and obtaining the images at each moment within a preset time length from the video stream in chronological order to form an image sequence specifically includes:

[0016] Receive in real time the video streams of the display interface collected by several cameras directed at different fixed angles toward the display interface of the operator workstation;

[0017] Obtain each frame of image within a preset time length from a plurality of video streams, and select a complete display interface image at each moment;

[0018] Extracting the corner points of the display from each selected frame of image and performing geometric transformation to correct the distortion and tilt of each frame of image;

[0019] A convolutional neural network is used to extract image features from each frame of the corrected image to eliminate image blur, distortion, and complex background interference;

[0020] Each frame of the image after the image features are extracted is arranged in time sequence to form the image sequence.

[0021] Preferably, the text recognition algorithm based on the convolutional recurrent neural network is used for the position of the image obtained by the detection to obtain the text result of the manual input control command of the operator operating the unit, which specifically includes:

[0022] A deep convolutional neural network is used to extract image features based on the location of the image obtained by detection;

[0023] A bidirectional long short-term memory network is used to obtain the text recognition character set and its probability distribution based on the extracted image features;

[0024] The connection time classification (CTC) loss function is used to remove redundant characters from the text recognition character set, and the character with the highest probability in the probability distribution is extracted as the text result of the manual input control command of the operator operating the unit.

[0025] Preferably, the real-time acquisition of the voice information of the vote counting of the operator workstation and the recognition of the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian specifically include:

[0026] Receive the vote counting voice information from the operator workstation collected by the microphone in real time;

[0027] The voting voice information is subjected to preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform to obtain Mel frequency cepstral coefficient MFCC feature parameters;

[0028] The MFCC feature parameters are input into an acoustic model and a language model obtained by pre-training a preset corpus using a connection time classification method CTC, and calculation and analysis are performed to obtain a text result of the control command notified by the operator to the guardian.

[0029] In a second aspect, the present invention provides a system for preventing human errors in a main control room of a nuclear power plant, comprising:

[0030] An image acquisition and analysis module is used to acquire an image sequence composed of images of the operator workstation display interface in real time, and detect and identify images in the image sequence to obtain a manual input control command for the operator to operate the unit;

[0031] A voice acquisition and analysis module, used for acquiring the voice information of the vote counting of the operator workstation in real time, and identifying the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian;

[0032] The comparison and warning module is connected to the image acquisition and analysis module and the voice acquisition and analysis module, and is used to compare the manually input control command with the notification control command, and issue a warning when the two are inconsistent.

[0033] Preferably, the image acquisition and analysis module specifically includes:

[0034] An image acquisition device for continuously acquiring images of the operator workstation display interface in real time; and,

[0035] An image analysis module, connected to the image acquisition device, for continuously receiving the display interface images in real time and arranging the image sequence in chronological order, and detecting and identifying the image sequence to obtain the manual input control command of the operator to operate the unit;

[0036] The voice acquisition and analysis module specifically includes:

[0037] A voice collection device for collecting the voice information of the vote counting at the operator workstation in real time; and,

[0038] A voice analysis module, connected to the voice collection device, for receiving the voice information of the vote counting in real time, and identifying the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian;

[0039] The comparison and warning module specifically includes:

[0040] a comparison module, connected to the image analysis module and the voice analysis module, for comparing the manually input control command with the notification control command, and issuing a warning message when the two are inconsistent; and,

[0041] The early warning device is connected to the comparison module and is used to receive the early warning information of the comparison device and issue an early warning according to the early warning information.

[0042] Preferably, the image acquisition device is specifically a camera, and the voice acquisition device is specifically a microphone.

[0043] Preferably, the image analysis module specifically includes:

[0044] An image preprocessing unit, connected to the camera, for receiving in real time a video stream of the operator workstation display interface acquired by the camera, so as to obtain from the video stream in chronological order an image sequence consisting of images at each moment within a preset time length;

[0045] An image detection unit, connected to the image preprocessing unit, is used to detect the image sequence using a pre-trained target detection algorithm based on a convolutional neural network to obtain the image and position of the manual input control command of the operator operating the unit;

[0046] The image recognition unit is connected to the image detection unit and is used to adopt a text recognition algorithm based on a convolutional recurrent neural network according to the location of the image obtained by detection, so as to obtain the text result of the manual input control command of the operator operating the unit.

[0047] Preferably, the image preprocessing unit specifically includes:

[0048] A receiving subunit, connected to the plurality of cameras, for receiving in real time video streams of the display interface respectively collected by the plurality of cameras directed toward the display interface of the operator workstation from different fixed angles;

[0049] A selection subunit, connected to the receiving subunit, for acquiring each frame of image within a preset time length from a plurality of video streams, and selecting a complete display interface image at each moment;

[0050] A correction subunit, connected to the selection subunit, for extracting display corner points from each selected frame of image and performing geometric transformation to correct the distortion and tilt of each frame of image;

[0051] A first image feature extraction subunit is connected to the correction subunit and is used to extract image features from each frame of the corrected image using a convolutional neural network to eliminate image blur, distortion, and complex background interference;

[0052] The arrangement subunit is connected to the image feature subunit and is used to arrange each frame of the image after the image features are extracted in time sequence to form an image sequence.

[0053] Preferably, the image recognition unit specifically includes:

[0054] A second image feature extraction subunit is used to extract image features using a deep convolutional neural network for the position of the image obtained by detection;

[0055] An image feature text recognition subunit, connected to the second image feature extraction subunit, for obtaining a text recognition character set and its probability distribution using a bidirectional long short-term memory network for the extracted image features;

[0056] The text result extraction subunit is connected to the image feature text recognition subunit, and is used to remove redundant characters from the text recognition character set using a connection time classification method (CTC) loss function, and extract the character with the highest probability in the probability distribution as the text result of the operator's manual input control command for operating the unit.

[0057] Preferably, the speech analysis module specifically includes:

[0058] A receiving unit, connected to the microphone, for receiving in real time the voice information of the operator's workstation collected by the microphone;

[0059] A speech feature extraction unit connected to the receiving unit, used for obtaining Mel frequency cepstral coefficient MFCC feature parameters by preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform of the vote counting speech information;

[0060] The speech recognition unit is connected to the speech feature extraction unit and is used to input the MFCC feature parameters into the acoustic model and language model obtained by pre-training the preset corpus using the connection time classification method CTC to perform calculation analysis, and obtain the text result of the notification control command notified by the operator to the guardian.

[0061] A third aspect of the present invention provides a nuclear power plant main control device, including an operator workstation and the nuclear power plant main control room anti-human error system as described above.

[0062] The method, system and main control equipment for preventing human errors in the main control room of a nuclear power plant provided by the present invention can obtain the input control commands and notification control commands of the operator workstation in real time. When the two are found to be inconsistent, an early warning can be issued in time to remind the operator, thereby preventing nuclear power plant operation safety accidents caused by human errors of the operator, thereby improving the safety of nuclear power plant control. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of a method for preventing human errors in a main control room of a nuclear power plant in an embodiment of the present invention;

[0064] Figure 2 It is a structural schematic diagram of a system for preventing human errors in a main control room of a nuclear power plant in an embodiment of the present invention;

[0065] Figure 3 It is a structural schematic diagram of a main control device of a nuclear power plant in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of the present invention.

[0067] In the description of the present invention, it should be noted that the directions or positional relationships indicated by “upper” and the like are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience and simplification of the description. They do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0068] In the description of the present invention, the terms “first” and “second” are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0069] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connect", "set", "install", "fix" and the like should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] In the description of the present invention, each unit and module involved may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure; the units and modules involved may be implemented by software or by hardware, for example, the units and modules may be located in a processor.

[0071] In the description of the present invention, the functions and steps marked in the flowcharts and block diagrams of the present invention may occur in an order different from that marked in the drawings if there is no conflict.

[0072] Embodiment 1:

[0073] like Figure 1 As shown, Embodiment 1 of the present invention provides a method for preventing human errors in a main control room of a nuclear power plant, the method comprising:

[0074] S11, acquiring in real time an image sequence consisting of images of the operator workstation display interface, and detecting and identifying images in the image sequence to obtain a manual input control command for the operator to operate the unit.

[0075] Specifically, in this embodiment, since the present invention aims to solve the problem of timely discovering the inconsistency between the control command manually input by the operator and the control command notified by the vote counting, it is necessary to determine what control command the operator is currently inputting. Since the input control command will be displayed in real time on the display interface of the operator's workstation, it is necessary to capture the display interface image after the operator inputs the control command. The display interface of the operator's workstation will display a large amount of information in real time, and it is impossible to accurately predict when the operator will input the operation information. This embodiment obtains in real time a sequence of images of the display interface arranged in chronological order within a certain period of time, and obtains the input control command of the operator by detecting and identifying the image sequence. The purpose of detection is to determine the image and position where the input control command is located, and the purpose of identification is to convert the information in the image for subsequent comparative analysis. Detection and identification are achieved through certain algorithms. Specifically, the image and position where the input control command is located can be determined by detection first, and then the detected image and position can be identified to obtain the input control command. Alternatively, each frame of the image sequence can be identified first, and then the content of the input control command can be detected and analyzed based on the identification result.

[0076] In an optional embodiment of the present invention, the real-time acquisition of an image sequence composed of operator workstation display interface images, and the detection and recognition of images in the image sequence to obtain a manual input control command for the operator to operate the unit, specifically includes:

[0077] receiving in real time a video stream of the operator workstation display interface acquired by a camera, and acquiring images at each moment within a preset time length from the video stream in chronological order to form an image sequence;

[0078] The image sequence is detected using a pre-trained convolutional neural network-based target detection algorithm to obtain the image and position of the manual input control command of the operator operating the unit;

[0079] A text recognition algorithm based on a convolutional recurrent neural network is used for the location of the image obtained by detection to obtain the text results of the manual input control commands of the operator operating the unit.

[0080] Specifically, in this embodiment, the operator's display interface is first monitored in real time by an external camera, a video stream is obtained by camera shooting, and each frame of the video stream is obtained in chronological order, that is, a corresponding image sequence is obtained; by detecting and identifying the image sequence, it is determined when the operator inputs what control command, that is, it is necessary to detect which images in the image sequence are images after the operator inputs the control command, and determine the position of the control command in these images, and then obtain the content of the input control command through image recognition. For this purpose, an image processing method based on deep learning is used to detect and identify the image sequence. First, the image where the input control command is located and the position in the image are detected by using a target detection algorithm based on a convolutional neural network. The algorithm is obtained through pre-training. The data set required for training is obtained by pre-collecting different image sequences under different input control command conditions, and marking the image where the input control command is located and the position where the input control command is located. After the algorithm is trained with the data set, it can accurately detect the control command input box currently operated by the operator; then, it is only necessary to identify the control command input box in the detected image, which reduces the calculation amount of image recognition, and can quickly and accurately obtain the recognition result of the input control command. At the same time, in order to facilitate subsequent comparative analysis, the recognition result of the input control command is output as a text result.

[0081] In an optional embodiment of the present invention, the real-time receiving of the video stream of the display interface when the operator operates the unit captured by the camera, and obtaining the images at each moment within a preset time length from the video stream in chronological order to form an image sequence specifically includes:

[0082] Receive in real time the video streams of the display interface collected by several cameras directed at different fixed angles toward the display interface of the operator workstation;

[0083] Obtain each frame of image within a preset time length from a plurality of video streams, and select a complete display interface image at each moment;

[0084] Extracting the corner points of the display from each selected frame of image and performing geometric transformation to correct the distortion and tilt of each frame of image;

[0085] A convolutional neural network is used to extract image features from each frame of the corrected image to eliminate image blur, distortion, and complex background interference;

[0086] Each frame of the image after the image features are extracted is arranged in time sequence to form the image sequence.

[0087] Specifically, in this embodiment, considering that the operator may block the display interface during operation, multiple cameras are used to collect display interface videos from different fixed angles, and an image sequence for identifying input control commands is formed by selecting unobstructed images taken by different cameras and combining them according to their respective time periods, so as to avoid the inability to correctly and quickly obtain input control commands due to occlusion; in order to make each frame of the image sequence displayed according to the same standard and with the same clarity, it is necessary to preprocess each frame of the image sequence, especially for the image sequence composed of multiple cameras, it is necessary to eliminate the image difference caused by the difference in shooting angles through preprocessing, therefore, the difference in shooting angles is determined by extracting the corner points of the operator's display in the image, and the image distortion and tilt caused by the difference in shooting angles are eliminated through geometric transformation, and the image feature extraction is also performed through a convolutional neural network to eliminate the problems of image blur, distortion, complex background interference, etc., so as to improve the efficiency and accuracy of subsequent calculations.

[0088] In an optional embodiment of the present invention, the text recognition algorithm based on the convolutional recurrent neural network is used for the location of the image obtained by the detection to obtain the text result of the manual input control command of the operator operating the unit, which specifically includes:

[0089] A deep convolutional neural network is used to extract image features based on the location of the image obtained by detection;

[0090] A bidirectional long short-term memory network is used to obtain the text recognition character set and its probability distribution based on the extracted image features;

[0091] The connection time classification (CTC) loss function is used to remove redundant characters from the text recognition character set, and the character with the highest probability in the probability distribution is extracted as the text result of the manual input control command of the operator operating the unit.

[0092] Specifically, in this embodiment, a text recognition algorithm based on a convolutional recurrent neural network is designed. After the network is trained using a public text recognition data set, it is used to recognize the content in the control command input box obtained by detection to obtain the text result of the input control command. The algorithm can achieve better recognition effect for recognizing text in images.

[0093] S12. Acquire the voice information of the vote counting at the operator workstation in real time, and identify the voice information of the vote counting to obtain the notification control command from the operator to the guardian.

[0094] Specifically, in this embodiment, the recognition of the control commands communicated by the operator through counting votes can be achieved directly by means of voice recognition. It should be noted that the counting voice information should be obtained simultaneously with the image sequence. Specifically, when the counting voice information is received, the image sequence within a certain period of time before and after the counting voice information is sent is detected, and whether the image sequence contains input control commands with the same content is detected and identified. It is also possible to detect whether there is corresponding voice counting information when the input control command is detected and identified, so as to avoid misjudgment caused by the lack of synchronization between the two.

[0095] In an optional embodiment of the present invention, the real-time acquisition of the voice information of the operator workstation and the recognition of the voice information of the operator to obtain the notification control command for the operator to notify the guardian specifically include:

[0096] Receive the vote counting voice information from the operator workstation collected by the microphone in real time;

[0097] The voting voice information is subjected to preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform to obtain Mel frequency cepstrum coefficient MFCC (Mel frequency cepstrum coefficient) feature parameters;

[0098] The MFCC feature parameters are input into an acoustic model and a language model obtained by pre-training a preset corpus using a connection time classification method CTC, and a text result of the control command notified by the operator to the guardian is obtained.

[0099] Specifically, in this embodiment, the operator's voice information of counting votes is collected through a microphone, and the voice information of counting votes is recognized as text, which is in the same output form as the input control command recognition result for easy comparison; speech recognition specifically includes three steps: feature extraction, filtering and noise reduction, and text conversion. First, feature extraction is performed on the voice information of counting votes to obtain speech feature information. Feature extraction includes preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform, etc. Among them, preprocessing includes pre-emphasis, framing and windowing to achieve the purpose of improving the frequency and continuity of speech, making the speech signal in the same frequency band, and performing frame processing. The preprocessed data is subjected to fast Fourier transform to obtain the spectrum of each frame, and then filtered through the Mel filter bank, and then subjected to MFCC feature parameters are obtained through numerical calculation and discrete cosine transform; then the MFCC feature parameters are filtered and denoised. The specific method of filtering and denoising can be to use a variety of filters such as low-pass filtering and band-pass filtering to analyze the interference signal, and then use the statistical characteristics of the interference and signal (such as autocorrelation function, power spectrum, etc.) to calculate the best estimation algorithm, and implement the denoising function through the algorithm. After denoising, it is convenient to obtain the speech recognition result more accurately, and this step can be omitted; finally, text conversion is performed, and the MFCC feature parameters are input into the preset acoustic model and language model obtained by using CTC to train the preset corpus for calculation and analysis, and the speech feature information after filtering and denoising is converted into readable text, thereby completing the speech recognition processing and obtaining the text result of the notification control command.

[0100] S13: Compare the manually input control command and the notification control command, and issue an early warning when the two are inconsistent.

[0101] Specifically, in this embodiment, after the input control command and the notification control command are both identified as text results, the text contents of the two are directly compared to see if they are consistent. A certain margin can be given to the method for determining whether the text contents of the two are consistent, allowing a certain error in the identification result. Under this error, it can be determined that the two point to the same device and execute the same instruction; when the operator's input control command is inconsistent with the notification control command, an early warning message is issued to the operator to prompt the operator to check whether the operation is incorrect, thereby avoiding safety accidents in the nuclear power plant caused by the operator's incorrect operation; the specific method of issuing the early warning may include issuing an audible alarm through a loudspeaker, or may include displaying an "inconsistent" prompt message on the display interface of the operator's workstation to warn the operator, and of course may also include displaying a "no abnormality" prompt message on the display interface when the comparison results are consistent and the operation is normal, and any other form of early warning method may also be used, as long as it is sufficient to attract the attention of the operator.

[0102] Embodiment 2:

[0103] like Figure 2 and 3 As shown, Embodiment 2 of the present invention provides a system 1 for preventing human errors in a main control room of a nuclear power plant, the system comprising:

[0104] The image acquisition and analysis module 11 is used to acquire in real time an image sequence composed of images on the display interface of the operator workstation 2, and detect and identify images in the image sequence to obtain a manual input control command for the operator to operate the unit;

[0105] The voice acquisition and analysis module 12 is used to acquire the voice information of the vote counting of the operator workstation 2 in real time, and recognize the voice information of the vote counting to obtain the notification control command of the operator to notify the guardian;

[0106] The comparison and warning module 13 is connected to the image acquisition and analysis module 11 and the voice acquisition and analysis module 12, and is used to compare the manually input control command with the notification control command, and issue a warning when the two are inconsistent.

[0107] In an optional embodiment of the present invention, Figure 3 As shown, the image acquisition and analysis module 11 specifically includes:

[0108] An image acquisition device 111 is used for real-time and continuous acquisition of the display interface image of the operator workstation 2; and,

[0109] An image analysis module 112, connected to the image acquisition device 111, is used to continuously receive the display interface images in real time and arrange the image sequence in chronological order, and detect and identify the image sequence to obtain the manual input control command of the operator to operate the unit;

[0110] The voice acquisition and analysis module 12 specifically includes:

[0111] A voice collection device 121, used for collecting the voice information of the vote counting of the operator workstation 2 in real time; and,

[0112] The voice analysis module 122 is connected to the voice collection device 121 and is used to receive the voice information of the vote counting in real time and recognize the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian;

[0113] The comparison and warning module 13 specifically includes:

[0114] A comparison module 131, connected to the image analysis module 112 and the voice analysis module 122, is used to compare the manually input control command with the notification control command, and issue a warning message when the two are inconsistent; and,

[0115] The early warning device 132 is connected to the comparison module 131, and is used to receive the early warning information of the comparison device 131 and issue an early warning according to the early warning information.

[0116] In an optional embodiment of the present invention, the image acquisition device 111 is specifically a camera, and the voice acquisition device 121 is specifically a microphone.

[0117] In an optional embodiment of the present invention, the image analysis module 112 specifically includes:

[0118] An image preprocessing unit, connected to the camera, for receiving in real time the video stream of the display interface of the operator workstation 2 acquired by the camera, so as to obtain from the video stream in chronological order an image sequence consisting of images at each moment within a preset time length;

[0119] An image detection unit, connected to the image preprocessing unit, is used to detect the image sequence using a pre-trained target detection algorithm based on a convolutional neural network to obtain the image and position of the manual input control command of the operator operating the unit;

[0120] The image recognition unit is connected to the image detection unit and is used to adopt a text recognition algorithm based on a convolutional recurrent neural network according to the location of the image obtained by detection, so as to obtain the text result of the manual input control command of the operator operating the unit.

[0121] In an optional embodiment of the present invention, the image preprocessing unit specifically includes:

[0122] A receiving subunit, connected to the plurality of cameras, for receiving in real time the video streams of the display interface respectively collected by the plurality of cameras directed toward the display interface of the operator workstation 2 from different fixed angles;

[0123] A selection subunit, connected to the receiving subunit, for acquiring each frame of image within a preset time length from a plurality of video streams, and selecting a complete display interface image at each moment;

[0124] A correction subunit, connected to the selection subunit, for extracting display corner points from each selected frame of image and performing geometric transformation to correct the distortion and tilt of each frame of image;

[0125] A first image feature extraction subunit is connected to the correction subunit and is used to extract image features from each frame of the corrected image using a convolutional neural network to eliminate image blur, distortion, and complex background interference;

[0126] The arrangement subunit is connected to the image feature subunit and is used to arrange each frame of the image after the image features are extracted in time sequence to form an image sequence.

[0127] In an optional embodiment of the present invention, the image recognition unit specifically includes:

[0128] A second image feature extraction subunit is used to extract image features using a deep convolutional neural network for the position of the image obtained by detection;

[0129] An image feature text recognition subunit, connected to the second image feature extraction subunit, for obtaining a text recognition character set and its probability distribution using a bidirectional long short-term memory network for the extracted image features;

[0130] The text result extraction subunit is connected to the image feature text recognition subunit, and is used to remove redundant characters from the text recognition character set using a connection time classification method (CTC) loss function, and extract the character with the highest probability in the probability distribution as the text result of the operator's manual input control command for operating the unit.

[0131] In an optional embodiment of the present invention, the speech analysis module 122 specifically includes:

[0132] A receiving unit, connected to the microphone, for receiving in real time the voice information of the operator workstation 2 collected by the microphone;

[0133] A speech feature extraction unit connected to the receiving unit, used for obtaining Mel frequency cepstral coefficient MFCC feature parameters by preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform of the vote counting speech information;

[0134] The speech recognition unit is connected to the speech feature extraction unit and is used to input the MFCC feature parameters into the acoustic model and language model obtained by pre-training the preset corpus using the connection time classification method CTC to perform calculation analysis, and obtain the text result of the notification control command notified by the operator to the guardian.

[0135] Specifically, in this embodiment, the provided nuclear power plant main control room human error prevention system 1 corresponds to the process of embodiment 1 and achieves the corresponding invention purpose. The system does not limit the specific implementation method. For example, the system can set a monitoring camera as the image acquisition device 111, set a microphone as the voice acquisition device 121, and set a speaker as one of the early warning devices 132 on the operator workstation 2. The image analysis module 112, the voice analysis module 122 and the comparison module 131 are set in the background server, and the server is connected to the display of the operator workstation 2 at the same time. The display is also used as one of the early warning devices 132. After the computing power of the server is used to identify and compare the received video image collected by the camera and the audio collected by the microphone, when the comparison result is inconsistent and the operation is abnormal, an early warning is issued through the speaker and the display; in addition, an independent device can also be used to be set near the operator workstation, and the device integrates the image acquisition device 111, the voice acquisition device 121, the early warning device 132 and the chip, and the chip is used to realize the functions of the image analysis module 112, the voice analysis module 122 and the comparison module 131.

[0136] Embodiment 3:

[0137] like Figure 3 As shown, embodiment 3 of the present invention provides a nuclear power plant main control device, including an operator workstation 2, and also includes a nuclear power plant main control room anti-human error system 1 as described in embodiment 2.

[0138] Specifically, in this embodiment, the operator workstation 2 is used for the operator to control the operation of the nuclear power unit, and the nuclear power plant main control room anti-human error system 1 is used to identify and prompt the operator of possible operational errors to reduce human errors in the operation and control process of the nuclear power unit and improve safety.

[0139] The method, system and main control equipment for preventing human errors in the main control room of a nuclear power plant provided in embodiments 1-3 of the present invention obtain the input control commands and notification control commands of the operator workstation in real time. When the two are found to be inconsistent, early warning information is promptly issued to remind the operator, thereby preventing nuclear power plant operation safety accidents caused by human factors of the operator, thereby improving the operation safety of the nuclear power plant.

[0140] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for preventing human errors in a main control room of a nuclear power plant, characterized in that: The method comprises: Acquire an image sequence composed of operator workstation display interface images in real time, and detect and identify images in the image sequence to obtain the operator's manual input control command for operating the unit. The real-time acquisition of an image sequence composed of operator workstation display interface images specifically includes: Receive in real time the video streams of the display interface collected by several cameras facing the operator workstation display interface from different fixed angles, Obtain each frame of image within a preset time length from several video streams, and select the complete display interface image at each moment. Extract the display corner points of each selected frame image and perform geometric transformation to correct the distortion and tilt of each frame image. A convolutional neural network is used to extract image features from each corrected image frame to eliminate image blur, distortion, and complex background interference. Arranging each frame of the image after extracting the image features in time sequence to form the image sequence; Acquire the voice information of the vote counting of the operator workstation in real time, and recognize the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian; The manually input control command and the notification control command are compared, and an early warning is issued when the two are inconsistent.

2. The method according to claim 1, characterized in that The detecting and identifying the images in the image sequence to obtain the manual input control command of the operator to operate the unit specifically includes: The image sequence is detected using a pre-trained convolutional neural network-based target detection algorithm to obtain the image and location of the manual input control command of the operator operating the unit; A text recognition algorithm based on a convolutional recurrent neural network is used for the location of the image obtained by detection to obtain the text results of the manual input control commands of the operator operating the unit.

3. The method according to claim 2, characterized in that The text recognition algorithm based on the convolutional recurrent neural network is used for the position of the image obtained by the detection to obtain the text result of the manual input control command of the operator operating the unit, which specifically includes: A deep convolutional neural network is used to extract image features based on the location of the image obtained by detection; A bidirectional long short-term memory network is used to obtain the text recognition character set and its probability distribution based on the extracted image features; The connection time classification (CTC) loss function is used to remove redundant characters from the text recognition character set, and the character with the highest probability in the probability distribution is extracted as the text result of the manual input control command of the operator operating the unit.

4. The method according to claim 1, characterized in that The real-time acquisition of the voice information of the vote counting of the operator workstation and the recognition of the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian specifically include: Receive the vote counting voice information from the operator workstation collected by the microphone in real time; The voting voice information is subjected to preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform to obtain Mel frequency cepstral coefficient MFCC feature parameters; The MFCC feature parameters are input into an acoustic model and a language model obtained by pre-training a preset corpus using a connection time classification method CTC, and calculation and analysis are performed to obtain a text result of the control command notified by the operator to the guardian.

5. A system for preventing human errors in the main control room of a nuclear power plant, characterized in that: include: The image acquisition and analysis module is used to acquire in real time an image sequence composed of images of the operator's workstation display interface, and detect and identify images in the image sequence to obtain the operator's manual input control command for operating the unit. The image acquisition and analysis module specifically includes an image analysis module, and the image analysis module specifically includes an image preprocessing unit, and the image preprocessing unit specifically includes: The receiving subunit is connected to a plurality of cameras and is used to receive in real time the video streams of the display interface collected by a plurality of cameras directed at different fixed angles toward the display interface of the operator workstation. A selection subunit is connected to the receiving subunit and is used to obtain each frame image within a preset time length from a plurality of video streams, and select a complete display interface image at each moment therefrom. A correction subunit, connected to the selection subunit, is used to extract the corner points of the display from each selected frame of image and perform geometric transformation to correct the distortion and tilt of each frame of image. The first image feature extraction subunit is connected to the correction subunit and is used to extract image features from each corrected frame of image using a convolutional neural network to eliminate image blur, distortion, and complex background interference. an arrangement subunit, connected to the image feature subunit, for arranging each frame of the image after the image features are extracted in time sequence to form an image sequence; A voice acquisition and analysis module, used for acquiring the voice information of the vote counting of the operator workstation in real time, and identifying the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian; The comparison and warning module is connected to the image acquisition and analysis module and the voice acquisition and analysis module, and is used to compare the manually input control command with the notification control command, and issue a warning when the two are inconsistent.

6. The system according to claim 5, characterized in that The image acquisition and analysis module specifically includes: An image acquisition device for continuously acquiring images of the operator workstation display interface in real time; and, An image analysis module, connected to the image acquisition device, for continuously receiving the display interface images in real time and arranging the image sequence in chronological order, and detecting and identifying the image sequence to obtain the manual input control command of the operator to operate the unit; The voice acquisition and analysis module specifically includes: A voice collection device for collecting the voice information of the vote counting at the operator workstation in real time; and, A voice analysis module, connected to the voice collection device, for receiving the voice information of the vote counting in real time, and identifying the voice information of the vote counting to obtain the notification control command notified by the operator to the guardian; The comparison and warning module specifically includes: a comparison module, connected to the image analysis module and the voice analysis module, for comparing the manually input control command with the notification control command, and issuing a warning message when the two are inconsistent; and, The early warning device is connected to the comparison module and is used to receive the early warning information of the comparison module and issue an early warning according to the early warning information.

7. The system according to claim 6, characterized in that The image acquisition device is specifically a camera, and the voice acquisition device is specifically a microphone.

8. The system according to claim 7, characterized in that The image analysis module specifically includes: An image preprocessing unit connected to the camera; An image detection unit, connected to the image preprocessing unit, is used to detect the image sequence using a pre-trained target detection algorithm based on a convolutional neural network to obtain the image and position of the manual input control command of the operator operating the unit; The image recognition unit is connected to the image detection unit and is used to adopt a text recognition algorithm based on a convolutional recurrent neural network according to the location of the image obtained by detection, so as to obtain the text result of the manual input control command of the operator operating the unit.

9. The system according to claim 8, characterized in that The image recognition unit specifically includes: A second image feature extraction subunit is used to extract image features using a deep convolutional neural network for the position of the image obtained by detection; An image feature text recognition subunit, connected to the second image feature extraction subunit, for obtaining a text recognition character set and its probability distribution using a bidirectional long short-term memory network for the extracted image features; The text result extraction subunit is connected to the image feature text recognition subunit, and is used to remove redundant characters from the text recognition character set using a connection time classification method (CTC) loss function, and extract the character with the highest probability in the probability distribution as the text result of the operator's manual input control command for operating the unit.

10. The system according to claim 7, characterized in that The speech analysis module specifically includes: A receiving unit, connected to the microphone, for receiving in real time the voice information of the operator's workstation collected by the microphone; A speech feature extraction unit connected to the receiving unit, used for obtaining Mel frequency cepstral coefficient MFCC feature parameters by preprocessing, fast Fourier transform, Mel filter bank filtering, logarithmic operation and discrete cosine transform of the vote counting speech information; The speech recognition unit is connected to the speech feature extraction unit and is used to input the MFCC feature parameters into the acoustic model and language model obtained by pre-training the preset corpus using the connection time classification method CTC to perform calculation analysis, and obtain the text result of the notification control command notified by the operator to the guardian.

11. A nuclear power plant main control equipment, including an operator workstation, characterized in that: It also includes a nuclear power plant main control room anti-human error system as described in any one of claims 5-10.

Images