Voice maintenance method and device, and storage medium

Abstract

The invention discloses a voice maintenance method and device and a storage medium. The method comprises the steps that it is determined that a client enters a voice auxiliary state when displaying a maintenance page of an elevator; receiving a maintenance auxiliary request sent by the client in the voice auxiliary state; the maintenance auxiliary request comprises a maintenance voice signal which is acquired by maintenance personnel and contains maintenance operation of the elevator; verifying the identity of a maintainer according to the maintenance voice signal; if the verification is successful, performing voice recognition on the maintenance voice signal to obtain maintenance operation information; the maintenance operation information is compiled into a maintenance operation instruction matched with the maintenance page of the elevator; and the maintenance operation instruction is sent to the client side, and the maintenance operation of the elevator is executed in the maintenance page of the elevator according to the maintenance operation instruction. According to the embodiment, the two hands of the maintenance personnel are liberated, the maintenance personnel continuously execute the maintenance operation of the elevator, and the maintenance efficiency of the elevator is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of elevators, and in particular to a voice maintenance method, device and storage medium. Background Art

[0002] As elevators become more and more popular in people's work and life, they run frequently and their parts are subject to certain wear and tear. To ensure their safe operation, elevators are regularly maintained.

[0003] During work, elevator maintenance personnel will carry maintenance mobile terminals, such as PDA (Personal Digital Assistant), mobile phones, etc., to view elevator information and submit corresponding data on the maintenance mobile terminals.

[0004] However, maintenance personnel may not be able to operate the maintenance mobile terminal due to some reasons (such as wearing gloves) during work. In order to operate the maintenance mobile terminal, they must stop the work in hand, resulting in low maintenance efficiency. Summary of the invention

[0005] In view of this, the present invention provides a voice maintenance method, device and storage medium for using voice assistance to perform elevator maintenance and improve the efficiency of elevator maintenance.

[0006] A first aspect of the present invention provides a voice maintenance method, comprising:

[0007] Make sure the client enters the voice assistance state when displaying the elevator maintenance page;

[0008] receiving a maintenance assistance request sent by the client in the voice assistance state; the maintenance assistance request includes a maintenance voice signal collected from a maintenance personnel and containing the maintenance operation of the elevator;

[0009] Verifying the identity of the maintenance personnel according to the maintenance voice signal;

[0010] If the verification is successful, the maintenance voice signal is subjected to voice recognition to obtain maintenance operation information;

[0011] Compiling the maintenance operation information into maintenance operation instructions adapted to the maintenance page of the elevator;

[0012] The maintenance operation instruction is sent to the client, so as to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction.

[0013] A second aspect of the present invention provides a voice maintenance device, comprising:

[0014] The voice assistance state determination module is used to determine whether the client enters the voice assistance state when displaying the elevator maintenance page;

[0015] A maintenance assistance request receiving module, used to receive a maintenance assistance request sent by the client in the voice assistance state; the maintenance assistance request includes a maintenance voice signal collected from a maintenance personnel and containing the maintenance operation of the elevator;

[0016] An identity verification module, used to verify the identity of the maintenance personnel based on the maintenance voice signal;

[0017] A maintenance operation information recognition module is used to perform voice recognition on the maintenance voice signal to obtain maintenance operation information if the verification is successful;

[0018] A maintenance operation instruction compiling module, used for compiling the maintenance operation information into maintenance operation instructions adapted to the maintenance page of the elevator;

[0019] A maintenance operation instruction sending module is used to send the maintenance operation instruction to the client, so as to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction.

[0020] A third aspect of the present invention provides an electronic device, the electronic device comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the voice maintenance method as described in the first aspect above.

[0024] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the voice maintenance method as described in the first aspect above is implemented.

[0025] A fifth aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the voice maintenance method as described in the first aspect above.

[0026] In this embodiment, it is determined that the client enters the voice assistance state when displaying the maintenance page of the elevator; a maintenance assistance request sent by the client in the voice assistance state is received; the maintenance assistance request includes a maintenance voice signal collected from the maintenance personnel and the content is the maintenance operation of the elevator; the identity of the maintenance personnel is verified based on the maintenance voice signal; if the verification is successful, the maintenance voice signal is voice recognized to obtain maintenance operation information; the maintenance operation information is compiled into a maintenance operation instruction adapted to the maintenance page of the elevator; the maintenance operation instruction is sent to the client to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction. In the maintenance scenario of the elevator, this embodiment uses the voice assistance function to assist the maintenance personnel in performing the maintenance operation of the elevator, which can ensure the accuracy of the maintenance operation of the elevator, free the hands of the maintenance personnel, allow the maintenance personnel to continue to perform the maintenance operation of the elevator, and improve the efficiency of elevator maintenance.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 This is a flow chart of a voice maintenance method provided in Example 1 of the present invention.

[0030] Figure 2 It is a structural diagram of a voice maintenance device provided in Embodiment 2 of the present invention.

[0031] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] See also Figure 1 , showing a flow chart of a voice maintenance method provided in Embodiment 1 of the present invention, the method can be executed by a voice maintenance device, the voice maintenance device can be implemented in the form of hardware and / or software, and the voice maintenance device can be configured in an electronic device.

[0036] like Figure 1 As shown, the method includes:

[0037] Step 101: Determine that the client enters a voice assistance state when displaying the elevator maintenance page.

[0038] In different types of buildings, there are different transportation needs for people, pets, goods, etc. Therefore, different types of elevators can be deployed in the buildings according to different transportation needs, such as passenger elevators, freight elevators, sightseeing elevators, etc.

[0039] Generally speaking, an elevator is a comprehensive system. Different types of elevators contain different components, and different maintenance strategies can be configured for different components.

[0040] In one example, the components of a certain type of elevator configuration include: an elevator control system, external call devices distributed on each floor, a car (including car doors), a motor for towing the car (also known as a traction machine), a control cabinet, a speed limiter, a door opener, a car frame, a car door, a counterweight guide rail, a car guide rail, a guide rail support, an accompanying cable, a counterweight device, a compensation chain (cable), a floor door, a guide device for the compensation chain (cable), a buffer, and the like.

[0041] In some types of elevators, components such as the traction machine, control cabinet, speed limiter, and accompanying cable can be omitted.

[0042] These components can be divided into different sets according to their functions, thus forming various subsystems that support the operation of the elevator. The elevator control system is wired to multiple systems of the elevator through serial ports or serial clock lines (SCL). The elevator control system monitors each system and controls the operation of each subsystem, so that the car moves in the hoistway and reaches each floor of the building.

[0043] In one example, the controller includes a door system, a frequency conversion system, a calling system and a traction system, wherein the door system is used to control the doors of the elevator, the elevator doors include car doors and hall doors of elevator waiting halls on each floor, the type of the car door is the same as the type of the hall door and they are switched (opened or closed) at the same time, the frequency conversion system is used to control the frequency converter, the calling system is used to control the logic of internal calls (calling the elevator in the car) and external calls (calling the elevator in the elevator waiting hall), and the traction system is used to control the movement of the car in the vertical direction (vertically upward or vertically downward) in the shaft.

[0044] Generally speaking, elevators are maintained in accordance with the statutory cycle:

[0045] 1. Half-month maintenance (every 15 days):

[0046] Check the traction machine oil level and brake clearance (≤0.7mm), test the light curtain response time (≤0.1 second), and clean the water in the pit.

[0047] Clean the interior facilities of the car (such as lighting, buttons), and lubricate the door system rails and hinges.

[0048] 2. Quarterly maintenance (every 3 months):

[0049] Adjust the verticality of the hall door (gap 1-6mm), check the door lock engagement length (≥7mm), and measure the traction rope tension deviation (≤5%).

[0050] Lubricate the door machine belt and test the speed limiter-safety clamp linkage.

[0051] 3. Half-year maintenance (every 6 months):

[0052] Replace the guide shoe liner that is worn more than 30% and test the insulation resistance of the control cabinet (power circuit ≥ 0.5MΩ).

[0053] Practice power outage emergency rescue procedures and ensure that the backup power supply lasts ≥ 30 minutes.

[0054] 4. Annual maintenance (comprehensive inspection):

[0055] Replace the gear oil of the traction machine reduction box (ISO VG220 standard) and check the wear of the safety clamp wedge (if it exceeds 15% of the original thickness, it needs to be replaced).

[0056] Adjust the speed deviation of the speed limiter (≤5%) and clean the brake core.

[0057] In addition, the maintenance of key components is also different. For example, for machine room equipment, check the operating status of the traction machine (no abnormal noise, temperature rise ≤85℃), lubricate the spindle head, ensure that the brake contact area is ≥80%, regularly clean the groove of the speed limiter and lubricate the shaft to ensure sensitive switch action; for the car and door system, clean the car floor and wall panels, check the emergency call button and overload alarm device function, lubricate the hall door guide rails and rollers, test the contact reliability of the door lock electrical contacts, for the shaft and pit, clean the shaft debris, check the flatness of the guide rails, the buffer oil level must meet the standard, and the water accumulated in the pit should be removed in time to prevent the buffer from rusting and failure, etc.

[0058] The manager or owner of the elevator can choose whether to install an edge computing node on the elevator based on factors such as the load status of the elevator. If you choose not to install an edge computing node, the controller maintains the original control logic and does not affect the normal operation of the elevator. If you choose to install an edge computing node, you can select a suitable computing device as the edge computing node of the elevator according to your needs. The edge computing node is combined with the original elevator control system into a new elevator control system and redefines the operation logic of the elevator.

[0059] Generally speaking, edge computing nodes are computing devices with strong computing capabilities, such as computers, servers or embedded devices. In addition, according to different intelligent services, edge computing nodes can be equipped with graphics processing units (GPUs) or embedded neural network processors (NPUs).

[0060] Edge computing nodes refer to a new business platform built at the edge of the network near the elevator, providing storage, computing, network and other resources, and sinking some key business applications to the edge of the access network to reduce the width and delay losses caused by network transmission and multi-level forwarding. The edge computing node is located between the user and the cloud (server), and is closer to the user (data source) than the traditional cloud. It has the characteristics of miniaturization, distribution and closer to the user. Massive data (such as audio data) no longer needs to be uploaded to the cloud for processing, and data processing is realized at the edge of the network, reducing request response time and network bandwidth while ensuring data security and privacy.

[0061] In addition, edge computing nodes can implement algorithm functions and model reasoning, communicate with the original elevator control system, and provide artificial intelligence (AI) and complex computing capabilities for the original elevator control system; edge computing nodes can also communicate with the cloud to implement algorithm functions and model updates, and transfer function calls of the original elevator control system.

[0062] In actual applications, maintenance personnel arrive at the elevator site with a maintenance mobile terminal, in which a client is installed. The maintenance personnel log in to the server using user account, password and other information. The server uses user account, password and other information to authenticate the client. When the authentication is successful, a communication connection is established between the client and the server.

[0063] When the maintenance personnel are maintaining the elevator, the client displays the elevator maintenance page, that is, the page related to the elevator maintenance work. When it is inconvenient for the maintenance personnel to manually operate the maintenance mobile terminal, the client can be controlled to enter the voice assistance state, that is, the state of using voice assistance to perform elevator maintenance operations.

[0064] In the specific implementation, one or more wake-up words can be set for the client. This wake-up word can be a default word or a user-defined word, for example, "Xiao Li", "Li Li", etc. The client calls the microphone of the maintenance mobile terminal to collect audio signals, and performs VAD (Voice Activity Detection) detection on the audio signals in real time. When a voice signal is detected (that is, someone is talking), the voice signal in the audio signal is intercepted and recorded as the wake-up voice signal. The wake-up voice signal is detected for the wake-up word using the KWS (Keyword Spotting) model and other methods. When it is detected that the wake-up voice signal matches the wake-up word, the voice assistance state is activated and the voice assistance state is synchronized to the server.

[0065] At this time, the server can determine that when the client displays the elevator maintenance page, it enters the voice assistance state because the wake-up voice signal collected by the maintenance personnel matches the wake-up word.

[0066] Step 102: Receive a maintenance assistance request sent by the client in the voice assistance state.

[0067] When the client enters the voice assistance state, it calls the speaker of the maintenance mobile terminal to play a response voice signal (such as "Please speak"), prompting the maintenance personnel to speak the content of the elevator maintenance operation for the current elevator maintenance page. At this time, the client can call the microphone of the maintenance mobile terminal to collect voice signals and perform VAD detection on the voice signals. When it is detected that the continuous silence (i.e., no voice signal) time exceeds a certain threshold (such as 1 second), the voice signal is saved and recorded as a maintenance voice signal.

[0068] The server provides an API (Application Programming Interface) to the client in the form of WebAPI (web page interface) and other methods. The client encapsulates the maintenance voice signal, additional information and other data into the maintenance assistance request, and calls the API to send the maintenance assistance request to the server.

[0069] Among them, the additional information includes the identity of the currently logged-in user (represented by the user account) (such as the identity ID), the task identifier of the maintenance operation (such as the task ID), the component information of each component in the maintenance page (such as the component name, component type and the list of operations supported by the component), the elevator's work number and real-time status data, etc. The additional information is saved in formats such as JSON (JavaScript Object Notation, JavaScript Object Notation).

[0070] For example, the additional information is as follows:

[0071]

[0072]

[0073]

[0074] In this example, the identity ID of the maintenance personnel is 00001111, the task ID of the maintenance operation is TASK_23G000001_20240110_01, and the maintenance page is "MAINPAGE"; the maintenance page contains components "level_precision", "device_no", "user", "repeat", "submit", and "elevator_status_query", each of which has name, type, and activation information, where name refers to the component name, type refers to the component type, and action refers to the list of operations that the component can support; timestamp refers to the timestamp of the current JSON data; the work number of the elevator is 23G000001, and the real-time status data of the elevator are brake_turn_distance (brake turning distance), brake_slipped_distance (brake slipping distance), ecar_slipped_distance (car slipping distance), open_door_time (opening time), close_door_time (closing time), etc. Different maintenance pages will carry different real-time status data of the elevator.

[0075] Step 103: Verify the identity of the maintenance personnel based on the maintenance voice signal.

[0076] In this embodiment, the maintenance assistance request includes a maintenance voice signal collected from the maintenance personnel and containing the maintenance operations of the elevator. When the maintenance assistance request sent by the client is received, the maintenance voice signal can be read from the maintenance assistance request, and the identity of the maintenance personnel to whom the maintenance voice signal belongs can be verified using the maintenance voice signal to ensure that the maintenance personnel who sends the maintenance voice signal is consistent with the user to whom the user account logged in by the client belongs, thereby preventing others from performing unauthorized operations through voice, improving the security of maintenance operation instructions, and thus ensuring the security of elevator maintenance operations.

[0077] In the specific implementation, the maintenance assistance request also includes the identity corresponding to the user account that logs in to the client. The user registers the standard voiceprint data to the server based on the user account. Then, the voiceprint model (such as speech_xvector_sv-zh-cn-cnceleb-16k-spk3465-pytorch, etc.) can be called to extract the target voiceprint data of the maintenance personnel from the maintenance voice signal, and query the standard voiceprint data configured for the identity.

[0078] A first similarity, such as cosine similarity, between the target voiceprint data and the standard voiceprint data is calculated, and the first similarity is compared with a preset first threshold.

[0079] If the first similarity is greater than or equal to a preset first threshold, it is determined that the identity authentication of the maintenance personnel is successful.

[0080] If the first similarity is less than a preset first threshold, it is determined that the identity authentication of the maintenance personnel has failed, and at this time, a failure result is sent to the client.

[0081] Step 104: If the verification is successful, voice recognition is performed on the maintenance voice signal to obtain maintenance operation information.

[0082] When the identity authentication of the maintenance personnel is successful, the speech recognition model (such as speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404, etc.) can be called to perform speech recognition on the maintenance voice signal, and convert the maintenance voice signal into maintenance operation information (text information) belonging to natural language.

[0083] Step 105: compile the maintenance operation information into maintenance operation instructions that are compatible with the elevator maintenance page.

[0084] In practical applications, the Large Language Model (LLM) can be called in the elevator maintenance scenario to read the key information of the maintenance operation from the maintenance operation information, and obtain one or more maintenance operation instructions adapted to the elevator maintenance page, that is, the maintenance operation instructions are used to perform the elevator maintenance operation in the elevator maintenance page.

[0085] In one embodiment of the present invention, step 105 may include the following steps:

[0086] Step 1051: Read the component identification representing the component in the maintenance operation information.

[0087] Step 1052: Read the operation type set for the component in the maintenance operation information.

[0088] Step 1053: Read the operation value set for the component in the maintenance operation information.

[0089] Step 1054: Combine the component identifier, operation type and operation value into a candidate operation instruction.

[0090] In this embodiment, a large language model can be called to read the component identifier that may represent the component, the operation type set for the component, and the operation value set for the component in the maintenance operation information. In some cases, the operation value can be empty or the default value. The component identifier, operation type, and operation value are combined to form a structured candidate operation instruction [Id-Action-Value] ([component identifier-operation type-operation value]).

[0091] For example, the maintenance operation information is "Enter 2.5 in the leveling accuracy box, enter 18G00001 in the elevator worker number box, select Zhang San in the user drop-down box, select the repeated execution item, then click the submit button, and finally voice broadcast the execution result." In the previous phrase "Enter 2.5 in the leveling accuracy box", the component identifier is the leveling accuracy box, the operation type is input, the operation value is 2.5, and the candidate operation instruction is [leveling accuracy box-input-2.5]. Similarly, other candidate operation instructions are [elevator worker number box-input-18G00001], [user drop-down box-select-Zhang San], [repeat execution item-select-true], [submit button-click-no value], [voice broadcast-execution-result].

[0092] Step 1055: Calibrate the candidate operation instructions according to the elevator maintenance page to obtain maintenance operation instructions.

[0093] Generally speaking, different maintenance personnel have different ways of speaking and naming, and the output format of the large language model is also not fixed. For example, for the "leveling accuracy box", some maintenance personnel may say "leveling accuracy", and some maintenance personnel may say "leveling"; for "input", some maintenance personnel may say "input", some maintenance personnel may say "fill in", and some maintenance personnel may say "fill in"; for "2.5", the large language model may output "two point five", and so on.

[0094] Therefore, the candidate operation instructions output by the large language model may not be directly executed in the elevator maintenance page. At this time, the candidate operation instructions can be calibrated according to the specifications of the elevator maintenance page, and the format can be unified to obtain maintenance operation instructions suitable for execution in the elevator maintenance page.

[0095] In actual applications, the maintenance assistance request also includes the component name and component type of each component in the elevator maintenance page. Then, a vectorization model (such as paraphrase-multilingual-MiniLM-L12-v2) can be used to vectorize the component name and component identifier of each component in the elevator maintenance page. If the vectorization is completed, the second similarity between the component identifier (vector) and the component name (vector) is calculated, such as cosine similarity.

[0096] The second similarities of the component names are compared, the second similarity with the highest value is screened out, and the second similarity with the highest value is compared with a preset second threshold (such as 0.7).

[0097] If the second similarity with the highest value is greater than or equal to a preset second threshold (such as 0.7), the component name is assigned to the component identifier, and the operation value Value is calibrated according to the requirements of the component type type. For example, when the component type type is numberic, the operation value Value is converted into a numerical value to obtain a maintenance operation instruction that is compatible with the elevator's maintenance page.

[0098] In addition, the maintenance assistance request also includes the elevator's work number and the real-time status data of the elevator that matches the maintenance page.

[0099] If the component type is input type (used to input numerical values), you can query multiple groups of sample status data [A1, A2, ..., An] recorded in the recent period of history (such as the last six months) of the elevator according to the work number. Each group of sample status data is adapted to the current maintenance page, which contains one or more status parameters and configuration sample values. The sample value is the value reasonably set for the specified component under the historical sample status data (represented by multiple status parameters) of the elevator. For example, A1 = [S11, S21, S31, ..., Sm1, C1], where S11-Sm1 is the status parameter under a group of sample status data, and C1 is the sample value.

[0100] For each group of sample state data, a third similarity between the real-time state data and the group of sample state data is calculated, such as Euclidean distance, cosine similarity, etc.

[0101] The third similarities of each group of sample status data are compared, and the sample value corresponding to the sample status data with the highest third similarity is used as the recommended input component value, which is written into the maintenance operation instruction for reference by maintenance personnel. At this time, the maintenance operation instruction is [Id-Action-Value-Suggestive] ([component identifier-operation type-operation value-sample value]).

[0102] Exemplarily, each state parameter in the real-time state data is aligned with each state parameter in the sample state data. If the alignment is completed, the difference between each state parameter in the real-time state data and each state parameter in the sample state data is squared to obtain the state deviation. The square root of the sum of each state deviation is taken to obtain the distance between the real-time state data and the sample state data to characterize the third similarity between the real-time state data and the sample state data.

[0103] In this example, the distance can be expressed as:

[0104]

[0105] Where D is the distance, m is the number of state parameters, i∈m, BSi is the i-th state parameter in the real-time state data, and ASi is the i-th state parameter in the sample state data.

[0106] Then, the distances between the groups of sample status data are compared, and the sample value corresponding to the sample status data with the smallest distance is used as the value of the recommended input component and written into the maintenance operation instruction.

[0107] Step 106: Send the maintenance operation instruction to the client, so as to perform the elevator maintenance operation in the elevator maintenance page according to the maintenance operation instruction.

[0108] In this embodiment, the server may send a maintenance operation instruction to the client, and the client may perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction.

[0109] In one case, if the maintenance operation instruction includes a component identifier, an operation type, and an operation value [Id-Action-Value], the maintenance operation instruction is sent to the client to locate the component whose component name is the component identifier in the elevator maintenance page, encapsulate the operation type and the operation value into a maintenance operation, and perform the maintenance operation on the component.

[0110] If the maintenance operation instruction includes component identifier, operation type, operation value and sample value [Id-Action-Value-Suggestive], the maintenance operation instruction is sent to the client to locate the component whose component name is the component identifier in the elevator maintenance page, encapsulate any one of the operation value and sample value and the operation type as a maintenance operation, and perform the maintenance operation on the component.

[0111] For example, the operation type and operation value can be encapsulated as a maintenance operation by default, and the maintenance personnel can be prompted with the sample value within a specified time (such as 5 seconds). When the maintenance personnel does not perform any operation, the maintenance operation is directly performed on the component. When the maintenance personnel selects the sample value through voice or other means, the sample value and operation type are encapsulated as a maintenance operation, and the maintenance operation is performed on the component.

[0112] In this embodiment, it is determined that the client enters the voice assistance state when displaying the maintenance page of the elevator; a maintenance assistance request sent by the client in the voice assistance state is received; the maintenance assistance request includes a maintenance voice signal collected from the maintenance personnel and the content is the maintenance operation of the elevator; the identity of the maintenance personnel is verified based on the maintenance voice signal; if the verification is successful, the maintenance voice signal is voice recognized to obtain maintenance operation information; the maintenance operation information is compiled into a maintenance operation instruction adapted to the maintenance page of the elevator; the maintenance operation instruction is sent to the client to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction. In the maintenance scenario of the elevator, this embodiment uses the voice assistance function to assist the maintenance personnel in performing the maintenance operation of the elevator, which can ensure the accuracy of the maintenance operation of the elevator, free the hands of the maintenance personnel, allow the maintenance personnel to continue to perform the maintenance operation of the elevator, and improve the efficiency of elevator maintenance.

[0113] Embodiment 2

[0114] See also Figure 2 , shows a schematic diagram of the structure of a voice maintenance device provided by Embodiment 2 of the present invention. Figure 2 As shown, the device comprises:

[0115] The voice assistance state determination module 201 is used to determine whether the client enters the voice assistance state when displaying the elevator maintenance page;

[0116] A maintenance assistance request receiving module 202 is used to receive a maintenance assistance request sent by the client in the voice assistance state; the maintenance assistance request includes a maintenance voice signal collected from a maintenance personnel and containing the maintenance operation of the elevator;

[0117] An identity verification module 203 is used to verify the identity of the maintenance personnel according to the maintenance voice signal;

[0118] The maintenance operation information recognition module 204 is used to perform voice recognition on the maintenance voice signal to obtain maintenance operation information if the verification is successful;

[0119] A maintenance operation instruction compiling module 205, used to compile the maintenance operation information into a maintenance operation instruction adapted to the maintenance page of the elevator;

[0120] The maintenance operation instruction sending module 206 is used to send the maintenance operation instruction to the client, so as to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction.

[0121] In one embodiment of the present invention, the voice assistance state determination module 201 includes:

[0122] A wake-up word setting module, used to set one or more wake-up words for the client;

[0123] The wake-up determination module is used to determine that when the client displays the elevator maintenance page, the wake-up voice signal collected from the maintenance personnel matches the wake-up word and enters the voice assistance state.

[0124] In one embodiment of the present invention, the maintenance assistance request also includes an identity identifier corresponding to a user account that logs in to the client;

[0125] The identity verification module 203 includes:

[0126] A target voiceprint data extraction module, used to extract the target voiceprint data of the maintenance personnel from the maintenance voice signal;

[0127] A standard voiceprint data query module, used to query the standard voiceprint data configured for the identity identifier;

[0128] A first similarity calculation module, used to calculate a first similarity between the target voiceprint data and the standard voiceprint data;

[0129] A verification success determination module, configured to determine that the identity verification of the maintenance personnel is successful if the first similarity is greater than or equal to a preset first threshold;

[0130] The verification failure determination module is used to determine that the identity verification of the maintenance personnel has failed if the first similarity is less than a preset first threshold.

[0131] In one embodiment of the present invention, the maintenance operation instruction compiling module 205 includes:

[0132] A component identification reading module, used for reading a component identification representing a component in the maintenance operation information;

[0133] An operation type reading module, used to read the operation type set for the component in the maintenance operation information;

[0134] An operation value reading module, used for reading the operation value set for the component in the maintenance operation information;

[0135] An operation instruction composition module, used for composing a candidate operation instruction from the component identifier, the operation type and the operation value;

[0136] An operation instruction calibration module is used to calibrate the candidate operation instructions according to the maintenance page of the elevator to obtain maintenance operation instructions.

[0137] In one embodiment of the present invention, the maintenance assistance request further includes the component name and component type of each component in the maintenance page of the elevator;

[0138] The operation instruction calibration module comprises:

[0139] A second similarity calculation module, used to calculate a second similarity between the component identifier and the component name;

[0140] A maintenance operation instruction generation module is used to assign the component name to the component identifier if the second similarity with the highest value is greater than or equal to a preset second threshold, and to calibrate the operation value according to the component type to obtain a maintenance operation instruction.

[0141] In one embodiment of the present invention, the maintenance assistance request further includes the work number of the elevator and the real-time status data of the elevator matching the maintenance page;

[0142] The operation instruction calibration module also includes:

[0143] A sample status data query module, for querying multiple groups of sample status data recorded in the history of the elevator according to the work number if the component type is an input type; each group of the sample status data is configured with a sample value;

[0144] A third similarity calculation module, used to calculate a third similarity between the real-time state data and the sample state data;

[0145] The recommended value writing module is used to write the sample value corresponding to the sample status data with the highest third similarity into the maintenance operation instruction as the recommended value to be input into the component.

[0146] In one embodiment of the present invention, the third similarity calculation module is further used for:

[0147] Squaring the difference between each state parameter in the real-time state data and each state parameter in the sample state data to obtain a state deviation;

[0148] Taking a square root of the sum of the state deviations to obtain a distance between the real-time state data and the sample state data, so as to represent a third similarity between the real-time state data and the sample state data;

[0149] The suggested value writing module is also used for:

[0150] The sample value corresponding to the sample status data with the smallest distance is used as a recommended value to be input into the component and written into the maintenance operation instruction.

[0151] In one embodiment of the present invention, the maintenance operation instruction sending module 206 is further used to:

[0152] If the maintenance operation instruction includes a component identifier, an operation type, and an operation value, the maintenance operation instruction is sent to the client to locate a component whose component name is the component identifier in the maintenance page of the elevator, encapsulate the operation type and the operation value into a maintenance operation, and perform the maintenance operation on the component;

[0153] If the maintenance operation instruction includes a component identifier, an operation type, an operation value and a sample value, the maintenance operation instruction is sent to the client to locate the component whose component name is the component identifier in the maintenance page of the elevator, encapsulate the operation value, any one of the sample values ​​and the operation type as a maintenance operation, and perform the maintenance operation on the component.

[0154] The voice maintenance device provided in the embodiment of the present invention can execute the voice maintenance method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the voice maintenance method.

[0155] Embodiment 3

[0156] See also Figure 3, shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0157] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0158] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the voice maintenance method.

[0160] In some embodiments, the voice maintenance method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the voice maintenance method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the voice maintenance method in any other appropriate manner (e.g., by means of firmware).

[0161] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0163] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0166] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0167] Embodiment 4

[0168] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the voice maintenance method provided by any embodiment of the present invention is implemented.

[0169] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0170] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0171] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A voice maintenance method, characterized in that: include: Make sure the client enters the voice assistance state when displaying the elevator maintenance page; Receiving a maintenance assistance request sent by the client in the voice assistance state; The maintenance assistance request includes a maintenance voice signal collected from the maintenance personnel and containing the maintenance operation of the elevator; Verifying the identity of the maintenance personnel according to the maintenance voice signal; If the verification is successful, the maintenance voice signal is subjected to voice recognition to obtain maintenance operation information; Compiling the maintenance operation information into maintenance operation instructions adapted to the maintenance page of the elevator; The maintenance operation instruction is sent to the client, so as to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction.

2. The method according to claim 1, characterized in that The step of determining that the client enters a voice assistance state when displaying the elevator maintenance page includes: Set one or more wake-up words for the client; It is determined that when the client displays the maintenance page of the elevator, the voice assistance state is entered because the wake-up voice signal collected from the maintenance personnel matches the wake-up word.

3. The method according to claim 1, characterized in that The maintenance assistance request also includes an identity identifier corresponding to the user account that logs in to the client; The verifying the identity of the maintenance personnel according to the maintenance voice signal includes: Extracting target voiceprint data of the maintenance personnel from the maintenance voice signal; Querying standard voiceprint data configured for the identity identifier; Calculating a first similarity between the target voiceprint data and the standard voiceprint data; If the first similarity is greater than or equal to a preset first threshold, it is determined that the identity verification of the maintenance personnel is successful; If the first similarity is less than a preset first threshold, it is determined that the identity authentication of the maintenance personnel has failed.

4. The method according to claim 1, characterized in that: The compiling the maintenance operation information into a maintenance operation instruction adapted to the maintenance page of the elevator includes: Reading a component identification representing a component in the maintenance operation information; Reading the operation type set for the component in the maintenance operation information; Reading an operation value set for a component in the maintenance operation information; Combining the component identifier, the operation type and the operation value into a candidate operation instruction; The candidate operation instructions are calibrated according to the maintenance page of the elevator to obtain maintenance operation instructions.

5. The method according to claim 4, characterized in that The maintenance assistance request also includes the component name and component type of each component in the maintenance page of the elevator; The step of calibrating the candidate operation instructions according to the maintenance page of the elevator to obtain a maintenance operation instruction adapted to the elevator includes: Calculating a second similarity between the component identifier and the component name; If the second similarity with the highest value is greater than or equal to a preset second threshold, the component name is assigned to the component identifier, and the operation value is calibrated according to the component type to obtain a maintenance operation instruction.

6. The method according to claim 5, characterized in that The maintenance assistance request also includes the work number of the elevator and the real-time status data of the elevator matching the maintenance page; The step of calibrating the candidate operation instruction according to the elevator to obtain the maintenance operation instruction further includes: If the component type is input type, multiple groups of sample status data recorded in the history of the elevator are queried according to the work number; each group of sample status data is configured with a sample value; Calculating a third similarity between the real-time status data and the sample status data; The sample value corresponding to the sample status data with the highest third similarity is used as a recommended value to be input into the component and written into the maintenance operation instruction.

7. The method according to claim 6, characterized in that The calculating a third similarity between the real-time status data and the sample status data comprises: Squaring the difference between each state parameter in the real-time state data and each state parameter in the sample state data to obtain a state deviation; Taking a square root of the sum of the state deviations to obtain a distance between the real-time state data and the sample state data, so as to represent a third similarity between the real-time state data and the sample state data; The step of taking the sample value corresponding to the sample status data having the highest third similarity as a value recommended to be input into the component and writing the value into the maintenance operation instruction includes: The sample value corresponding to the sample status data with the smallest distance is used as a recommended value to be input into the component and written into the maintenance operation instruction.

8. The method according to any one of claims 1 to 7, characterized in that The sending the maintenance operation instruction to the client so as to perform the maintenance operation of the elevator in the maintenance page of the elevator according to the maintenance operation instruction includes: If the maintenance operation instruction includes a component identifier, an operation type, and an operation value, the maintenance operation instruction is sent to the client to locate a component whose component name is the component identifier in the maintenance page of the elevator, encapsulate the operation type and the operation value into a maintenance operation, and perform the maintenance operation on the component; If the maintenance operation instruction includes a component identifier, an operation type, an operation value and a sample value, the maintenance operation instruction is sent to the client to locate the component whose component name is the component identifier in the maintenance page of the elevator, encapsulate the operation value, any one of the sample values ​​and the operation type as a maintenance operation, and perform the maintenance operation on the component.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the voice maintenance method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the voice maintenance method according to any one of claims 1 to 8 is implemented.

Images