Audio control method, device, vehicle and storage medium

By receiving and identifying the key operation of the steering wheel control system in the on-board system and processing the audio control operation separately, the problem of poor audio control response in the steering wheel control system is solved, and faster audio data response and more stable audio transmission are achieved .

CN118588078BActive Publication Date: 2024-11-15CHINA FAW CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411068439.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-15
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

When using the steering wheel control system, when users connect to the vehicle system through Bluetooth technology for audio control, communication delay problems often occur, resulting in poor responsiveness and affecting the user experience.

Method used

By receiving key operations from the side control system in the on-board system, task type recognition is performed, and audio control operations are sent separately to the mobile terminal. Bluetooth audio processing threads are used to process audio-type tasks, avoid task queuing, and improve the responsiveness of audio control.

Benefits of technology

Improves the responsiveness of audio control, reduces audio data transmission delay, improves audio transmission quality and stability, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118588078B_ABST
    Figure CN118588078B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide an audio control method, device, vehicle and storage medium, which belong to the field of vehicle audio control technology. The method includes: receiving key operations transmitted by a control system; identifying the task type of the key operations to obtain a target task type; filtering out audio control operations from the key operations according to the target task type; sending the audio control operations to a mobile terminal; receiving target audio information sent by the mobile terminal according to the audio control operations; wherein the target audio information includes audio data and audio attributes of the audio data; and performing audio control on preset audio playback information according to the audio data and audio attributes. The embodiments of the present application identify the task type of the key operations output by the control system, filter out the audio control operations, and can respond to the audio control operations separately without waiting for other tasks to be completed before executing the audio control operations, thereby improving the responsiveness of the audio control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of vehicle-mounted audio control, and in particular to an audio control method, device, vehicle and storage medium. Background Art

[0002] The steering wheel control system, also known as the square control system, arranges a series of function buttons on the steering wheel, allowing users to conveniently control various functions of the vehicle, such as navigation control, audio playback, and phone calls. In use, users usually connect their mobile devices to the vehicle system through Bluetooth technology to achieve functions such as audio playback and voice calls.

[0003] However, in actual application scenarios, when users control the audio playback through the square control system, there are often obvious communication delay problems and poor responsiveness, which directly affects the user experience. Therefore, how to improve the responsiveness of audio control has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an audio control method, device, vehicle and storage medium, aiming to improve the responsiveness of audio control.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes an audio control method, which is applied to a vehicle-mounted system. The method includes:

[0006] The receiving party controls the key operations transmitted by the system;

[0007] Performing task type identification on the key operation to obtain a target task type;

[0008] Filtering audio control operations from the key operations according to the target task type;

[0009] Sending the audio control operation to a mobile terminal;

[0010] Receiving target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data;

[0011] Audio control is performed on preset audio playing information according to the audio data and the audio attribute; wherein the audio playing information includes at least one of the following: playing audio and playing content.

[0012] In some embodiments, the vehicle-mounted system is used to connect the mobile terminal to the vehicle-mounted system via Bluetooth, the vehicle-mounted system includes a Bluetooth protocol stack, the Bluetooth protocol stack includes a plurality of Bluetooth threads, one of the Bluetooth threads is a Bluetooth audio processing thread;

[0013] The sending the audio control operation to the mobile terminal comprises:

[0014] Storing the audio control operation in the Bluetooth audio processing thread;

[0015] The audio control operation is sent to the mobile terminal through the Bluetooth audio processing thread, wherein the audio control operation is used to enable the mobile terminal to generate the target audio information.

[0016] In some embodiments, the step of performing task type identification on the key operation to obtain a target task type includes:

[0017] Converting the key operation into a key signal;

[0018] Performing key value conversion on the key signal to obtain key value data;

[0019] The key-value data is task classified to obtain the target task type.

[0020] In some embodiments, converting the key signal into a key value to obtain key value data includes:

[0021] Acquire a key mapping relationship table from the party control system;

[0022] A key value query is performed on the key signal based on the key mapping relationship table to obtain the key value data.

[0023] In some embodiments, the step of performing task classification on the key-value data to obtain the target task type includes:

[0024] Obtaining a preset key task mapping table; wherein the key task mapping table stores the task type corresponding to each key value data;

[0025] A task query is performed on the button key value data based on the button task mapping table to obtain the target task type.

[0026] In some embodiments, performing audio control on preset audio playback information according to the audio data and the audio attribute includes:

[0027] Controlling the playback of the audio according to the audio data;

[0028] The audio information of the playback content is updated based on the audio attribute.

[0029] To achieve the above object, a second aspect of an embodiment of the present application proposes an audio control method, which is applied to a mobile terminal. The method includes:

[0030] Receiving an audio control operation sent by the vehicle-mounted system; wherein the audio control operation is obtained by the method described in the first aspect above;

[0031] Perform audio switching based on the audio control operation to obtain target audio information;

[0032] The target audio information is sent to the vehicle-mounted system.

[0033] In some embodiments, performing audio switching based on the audio control operation to obtain target audio information includes:

[0034] Performing audio switching according to the audio control operation, and feeding back audio switching information to the in-vehicle system;

[0035] receiving an audio information acquisition instruction sent by the vehicle-mounted system according to the audio switching information;

[0036] Information acquisition is performed based on the audio information acquisition instruction to obtain the target audio information.

[0037] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application provides an audio control device, which is applied to a vehicle-mounted system. The device includes:

[0038] A key operation receiving module is used to receive key operations transmitted by the party control system;

[0039] A task type identification module, used to identify the task type of the key operation to obtain a target task type;

[0040] A control operation screening module, used for screening audio control operations from the key operations according to the target task type;

[0041] A control operation sending module, used to send the audio control operation to a mobile terminal;

[0042] A control information receiving module, configured to receive target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data;

[0043] The audio control module is used to perform audio control on preset audio playing information according to the audio data and the audio attribute; wherein the audio playing information includes at least one of the following: playing audio, playing content.

[0044] To achieve the above-mentioned purpose, a fourth aspect of an embodiment of the present application provides an audio control device, which is applied to a mobile terminal, and the device includes:

[0045] A control operation receiving module, used to receive an audio control operation sent by the vehicle-mounted system; wherein the audio control operation is obtained by the method described in the first aspect above;

[0046] An audio switching module, used to perform audio switching based on the audio control operation to obtain target audio information;

[0047] The audio information sending module is used to send the target audio information to the vehicle-mounted system.

[0048] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a vehicle, the vehicle comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method described in the first aspect when executing the computer program.

[0049] To achieve the above-mentioned purpose, the sixth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect above, or the method described in the second aspect above.

[0050] The audio control method, device, vehicle and storage medium proposed in the present application receive the key operation transmitted by the control system; identify the task type of the key operation to obtain the target task type; filter out the audio control operation from the key operation according to the target task type; send the audio control operation to the mobile terminal; receive the target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data; perform audio control on the preset audio playback information according to the audio data and the audio attributes. The embodiment of the present application identifies the task type of the key operation output by the control system, filters out the audio control operation, and can respond to the audio control operation alone without waiting for other tasks to be completed before executing the audio control operation, thereby improving the responsiveness of the audio control. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the communication architecture provided by an embodiment of the present application;

[0052] Figure 2 is a flow chart of the audio control method provided in an embodiment of the present application;

[0053] Figure 3 yes Figure 2 Flow chart of step S102 in FIG.

[0054] Figure 4 yes Figure 3 Flow chart of step S202 in FIG.

[0055] Figure 5 yes Figure 3 Flow chart of step S203 in FIG.

[0056] Figure 6 yes Figure 2 Flow chart of step S104 in FIG.

[0057] Figure 7 yes Figure 2 Flow chart of step S106 in FIG.

[0058] Figure 8 is a flow chart of an audio control method provided by another embodiment of the present application;

[0059] Fig. 9 yes Figure 8 Flow chart of step S702 in FIG.

[0060] Fig.10 is a schematic diagram of the structure of an audio control device provided in an embodiment of the present application;

[0061] Fig.11 is a structural schematic diagram of an audio control device provided by another embodiment of the present application;

[0062] Fig.12 It is a schematic diagram of the hardware structure of the vehicle provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] LIN communication, LIN communication is a serial communication protocol for vehicle networks, used for inter-module communication in automotive electronic systems. LIN communication is designed for transmitting small amounts of data within the vehicle and is typically used to connect modules such as door controllers, sunroof controllers, and seat controllers. The LIN protocol can reduce costs and improve efficiency while maintaining reliability.

[0068] Bluetooth protocol stack, Bluetooth protocol stack is an open specification for wireless data and digital communication. It is the core component of Bluetooth technology. It enables devices to locate each other and establish connections. Through Bluetooth connection, devices can interact and exchange data. The Bluetooth protocol stack includes the physical layer, link layer, Bluetooth manager, L2CAP (Logical Link Control and Adaptation Layer) and various Bluetooth profiles. These protocol layers work together to enable Bluetooth devices of different brands and types to connect to each other and exchange data.

[0069] In-vehicle system refers to a series of electronic systems and devices installed inside the car. It integrates advanced information technology, communication technology, control technology, etc. to provide drivers and passengers with a convenient, safe and comfortable driving experience. The in-vehicle system has a variety of functions, such as navigation, communication, entertainment, information services, vehicle control and safety, etc., aiming to improve the intelligence and automation level of the car and meet the needs of modern people for high-quality travel.

[0070] The steering wheel control system, also known as the square control system, arranges a series of function buttons on the steering wheel, allowing the driver to conveniently control various functions of the vehicle, such as navigation control, audio playback, making phone calls, etc.

[0071] With the continuous development of smart cars, smart cockpits are becoming more and more intelligent. Many functions are realized through soft switches. However, when turning on or adjusting related functions during driving, if you click the soft switch from the central control, it will affect driving safety. Therefore, through the buttons on the square control system, the driver can directly control many electronic devices, which improves driving safety.

[0072] In actual application scenarios, users usually connect their mobile devices to the vehicle system through Bluetooth technology to achieve functions such as audio playback and voice calls. While driving, when the driver uses the control system to control the audio played by the mobile device via Bluetooth, there will often be obvious communication delay problems. There may be a probability that the mobile terminal has already played the next song, but the vehicle system has just executed the switching control. The responsiveness is poor, which directly affects the user experience.

[0073] Based on this, the embodiments of the present application provide an audio control method, device, vehicle and storage medium, aiming to improve the responsiveness of audio control.

[0074] The audio control method, device, vehicle and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the audio control method in the embodiments of the present application is described.

[0075] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0076] See also Figure 1 , Figure 1 The communication architecture diagram provided for the embodiment of the present application includes a vehicle-mounted system, a party control system and a mobile terminal, and the vehicle-mounted system is communicatively connected with the party control system and the mobile terminal.

[0077] In some embodiments, the vehicle-mounted system and the mobile terminal are connected to each other via Bluetooth, and the Bluetooth audio can be controlled, including but not limited to: play, pause, switch audio and other operations.

[0078] The square control system communicates with the vehicle system via LIN communication.

[0079] In some embodiments, the square control system can also be connected to the vehicle system through CAN bus, Ethernet, etc., but is not limited to this.

[0080] In some embodiments, the in-vehicle system may be an in-vehicle entertainment system, such as a central control car computer, or an in-vehicle navigation system, etc., but is not limited thereto.

[0081] In some embodiments, a mobile terminal refers to a computer device that can be used on the move, and the mobility of the mobile terminal is mainly reflected in mobile communication capabilities and portable size. The mobile terminal may include a mobile phone, a smart phone, a notebook, a tablet computer, etc., but is not limited thereto.

[0082] It should be noted that the audio control method can be applied to a vehicle-mounted system or a mobile terminal, which will be specifically described in the following embodiments.

[0083] In some embodiments, the audio control method applied to the vehicle-mounted system is described by the following embodiments.

[0084] See also Figure 2 , Figure 2 is an optional flow chart of the audio control method provided in the embodiment of the present application, which is applied to the vehicle-mounted system. Figure 2 The method may include but is not limited to steps S101 to S106.

[0085] Step S101, receiving a key operation transmitted by a control system;

[0086] Step S102, identifying the task type of the key operation to obtain the target task type;

[0087] Step S103, filtering audio control operations from key operations according to the target task type;

[0088] Step S104, sending the audio control operation to the mobile terminal;

[0089] Step S105, receiving target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data;

[0090] Step S106, performing audio control on the preset audio playing information according to the audio data and the audio attribute; wherein the audio playing information includes at least one of the following: playing audio, playing content.

[0091] In steps S101 to S106 shown in the embodiment of the present application, task types are identified through key operations output by the remote control system, audio control operations are screened out, and the audio control operations are responded to separately. There is no need to wait for other tasks to be completed before executing the audio control operations, thereby improving the responsiveness of the audio control.

[0092] In step S101 of some embodiments, the key operations transmitted by the party control system may include, but are not limited to: audio control, navigation control, telephone control, display screen interface switching, etc., but are not limited thereto.

[0093] See also Figure 3 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0094] Step S201, converting the key operation into a key signal;

[0095] Step S202, converting the key signal into a key value to obtain key value data;

[0096] Step S203, performing task classification on the key value data to obtain a target task type.

[0097] Steps S201 to S203 shown in the embodiment of the present application can determine the user's operation intention from numerous key operations by performing signal conversion, key value conversion and task classification on key operations, which helps to better respond to user needs.

[0098] In step S201 of some embodiments, the party control system and the vehicle-mounted system are connected to each other through LIN communication. Therefore, the user's key operation behavior is first converted into a LIN signal for communication and transmission.

[0099] After receiving the key signal converted into the LIN signal, the key signal needs to be converted into a key value so that the system can recognize and understand it.

[0100] See also Figure 4 In some embodiments, step S202 may include but is not limited to steps S301 to S302:

[0101] Step S301, obtaining a key mapping relationship table from the party control system;

[0102] Step S302: perform a key value query on the key signal based on the key mapping relationship table to obtain key value data.

[0103] Steps S301 to S302 shown in the embodiment of the present application can quickly and accurately convert key signals into corresponding key value data through the key mapping relationship table obtained from the party control system, thereby accelerating the system's response speed to user input, and different terminals can uniformly process key signals to enhance interoperability between terminals.

[0104] In step S301 of some embodiments, the button mapping relationship table of the square control system may be pre-established by a technician during the research and development process, or may be pre-defined by a user during daily use, but is not limited thereto.

[0105] In step S302 of some embodiments, key-value query can be implemented through linear search, index search, tree structure search, etc., and the specific search method needs to be selected based on factors such as the number of vehicle button functions, response time requirements, processor performance, etc., and is not limited in the embodiments of the present application.

[0106] In some embodiments, the key mapping relationship table can refer to Table 1, which is a key mapping relationship table. By performing a key value query in the key mapping relationship table through a key signal, the key value data corresponding to the key signal can be obtained.

[0107] Table 1

[0108]

[0109] It should be noted that Table 1 is only for illustration and is not intended to limit the present application.

[0110] See also Figure 5 In some embodiments, step S203 may include but is not limited to steps S401 to S402:

[0111] Step S401, obtaining a preset key task mapping table; wherein the key task mapping table stores the task type corresponding to each key value data;

[0112] Step S402: perform task query on the key value data based on the key task mapping table to obtain the target task type.

[0113] In steps S401 to S402 shown in the embodiment of the present application, a task query is performed on the key value signal through a pre-set key task mapping table to obtain the task type corresponding to each key operation, so that the function corresponding to each key operation can be quickly found, thereby controlling more efficiently and improving the accuracy of system recognition.

[0114] In step S401 of some embodiments, the key task mapping table may be pre-established by a technician during the research and development process, or may be pre-defined by a user during daily use, but is not limited thereto.

[0115] In step S402 of some embodiments, task query can be implemented through linear search, index search, tree structure search, etc., and the specific search method needs to be selected according to factors such as the number of vehicle button functions, response time requirements, processor performance, etc., and is not limited in the embodiments of the present application.

[0116] In some embodiments, the key task mapping table can refer to Table 2, which is the key task mapping table. Based on the key value data, a task query is performed in the key task mapping table to obtain the task type corresponding to the key value data.

[0117] Table 2

[0118]

[0119] It should be noted that Table 2 is only for illustration and is not intended to limit the present application.

[0120] In step S103 of some embodiments, since the user's key pressing behavior on the square control system may generate more than one key operation signal, the system needs to filter out the audio control operation from the key operation to facilitate subsequent targeted and separate response to the audio control operation, thereby improving the responsiveness of the audio control.

[0121] It should be noted that the audio control operations may include but are not limited to: play, pause, switch, jump, volume adjustment, etc., but are not limited to these.

[0122] In some embodiments, the vehicle-mounted system is provided with a Bluetooth protocol stack, and the Bluetooth protocol stack is used to implement a Bluetooth communication connection between the vehicle-mounted system and the mobile terminal, so that data can be exchanged.

[0123] Among them, the Bluetooth protocol stack includes multiple Bluetooth threads, one of which is a Bluetooth audio processing thread. The Bluetooth audio processing thread is used to process audio tasks and is distinguished from other types of Bluetooth tasks, thereby avoiding the queuing of tasks in the Bluetooth thread and solving the communication delay problem during single-threaded Bluetooth protocol stack processing, thereby improving the responsiveness of audio control.

[0124] It should be noted that the Bluetooth audio processing thread and other Bluetooth threads in the Bluetooth protocol stack read and write the same memory area. When setting the thread, different sizes of memory space will be allocated to each thread to ensure that each Bluetooth thread can work normally.

[0125] In some embodiments, the number of tasks that can be processed simultaneously by each Bluetooth thread may be limited, for example, each Bluetooth thread is limited to executing at most two tasks simultaneously, thereby ensuring that resources can meet task requirements.

[0126] See also Figure 6 In some embodiments, step S104 may include but is not limited to steps S501 to S502:

[0127] Step S501, storing the audio control operation in the Bluetooth audio processing thread;

[0128] Step S502: sending an audio control operation to the mobile terminal through the Bluetooth audio processing thread, wherein the audio control operation is used to enable the mobile terminal to generate target audio information.

[0129] Steps S501 to S502 shown in the embodiment of the present application process the audio control operation through a newly set Bluetooth audio processing thread, which helps to improve the response speed and real-time performance of audio control, reduce interference with other threads, effectively reduce the delay of audio data transmission, improve audio transmission quality, and ensure the stability and continuity of audio communication.

[0130] In some embodiments, the vehicle-mounted system also includes an intermediate layer, wherein all key operations, signals, and instruction transmissions are executed through the intermediate layer.

[0131] It should be noted that the middle layer, also known as the business logic layer, is responsible for processing the business logic and rules of the application, which can ensure the uniformity and consistency of business rules and simplify the modification and maintenance of business rules.

[0132] In step S105 of some embodiments, the target audio information includes audio data and audio attributes of the audio data; wherein the audio data may be music data, podcast data, video audio data, etc., but is not limited thereto.

[0133] The audio attributes of the audio data may include, but are not limited to, the creator of the audio, the lyrics / lines of the audio, the total duration of the audio, the playback progress of the audio, etc.

[0134] In some embodiments, the vehicle-mounted system further includes a display unit, which is used to display audio-related information, that is, preset audio playback information, wherein the audio playback information includes: playback audio, playback content;

[0135] The playback content may include playback progress, playback lyrics, playback mode, creator information and other information.

[0136] See also Figure 7 In some embodiments, step S106 may also include but is not limited to steps S601 to S602:

[0137] Step S601, controlling the playing of the audio according to the audio data;

[0138] Step S602: updating audio information of the playback content based on the audio attributes.

[0139] Steps S601 to S602 shown in the embodiment of the present application control the playback of audio according to audio data and update the audio content on the display unit based on audio attribute information, which helps to improve the user experience of the vehicle system. By adjusting the audio playback in real time to match the audio attributes, the user can obtain a clearer user experience and can more intuitively understand the currently playing audio content, thereby improving the comfort and safety during driving.

[0140] In step S601 of some embodiments, the audio playback control includes playing audio and pausing audio.

[0141] It should be noted that the action of switching audio operation is implemented by the mobile terminal. If the mobile terminal switches the audio being played, the returned target audio information is the relevant information of the switched audio, and the vehicle-mounted system is only responsible for the play and pause operations.

[0142] In some embodiments, the audio attributes include the creator of the audio, the lyrics / lines of the audio, the total duration of the audio, and the playback progress of the audio;

[0143] The playback content may include playback progress, playback lyrics, creator information, and other information.

[0144] In some embodiments, step S602 may include but is not limited to the following steps:

[0145] The playing progress on the display unit is updated based on the total duration of the audio and the playing progress of the audio in the audio properties;

[0146] Update the playback lyrics on the display unit based on the playback lyrics in the audio attributes;

[0147] The creator information on the display unit is updated based on the audio creator in the audio attributes.

[0148] Thereby, the audio information and the played audio can be matched.

[0149] In some embodiments, the audio control method applied to a mobile terminal is described by the following embodiments.

[0150] See also Figure 8 , Figure 8 is an optional flow chart of an audio control method provided by another embodiment of the present application, which is applied to a mobile terminal. Figure 8 The method may include but is not limited to steps S701 to S703.

[0151] Step S701, receiving an audio control operation sent by the vehicle system;

[0152] Step S702, performing audio switching based on the audio control operation to obtain target audio information;

[0153] Step S703: Send the target audio information to the vehicle-mounted system.

[0154] Steps S701 to S703 shown in the embodiment of the present application perform audio switching through the mobile terminal according to the received audio control instructions, and return the switched audio information to the vehicle system, which is beneficial to improving the driver's concentration and safety, enabling the driver to more conveniently control the vehicle audio system and enhance the driving experience.

[0155] In step S701 of some embodiments, the audio control operation is performed by Figures 2 to 7 Any of the embodiments shown is obtained.

[0156] See also Fig. 9In some embodiments, step S702 may include but is not limited to steps S801 to S803:

[0157] Step S801, performing audio switching according to the audio control operation, and feeding back audio switching information to the vehicle system;

[0158] Step S802, receiving an audio information acquisition instruction sent by the vehicle system according to the audio switching information;

[0159] Step S803: acquiring information based on the audio information acquisition instruction to obtain target audio information.

[0160] Steps S801 to S803 shown in the embodiment of the present application improve the interactive logic between the vehicle-mounted system and the mobile terminal through information feedback and instruction reception, making information acquisition and transmission more convenient, safe and efficient.

[0161] In step S801 of some embodiments, audio switching includes but is not limited to operations such as previous song, next song, play, and pause.

[0162] In a specific embodiment, it includes a vehicle-mounted system, a square control system and a mobile terminal. The vehicle-mounted system and the mobile terminal are connected to each other through Bluetooth communication, so that the Bluetooth audio can be controlled; the square control system and the vehicle-mounted system are connected to each other through LIN communication.

[0163] Specifically, the vehicle-mounted system is provided with an intermediate layer, a Bluetooth protocol stack, and a display unit;

[0164] Among them, the middle layer is used to realize the transmission of all key operations, signals, and commands in the vehicle system;

[0165] The Bluetooth protocol stack is used to realize Bluetooth communication connection between the vehicle system and the mobile terminal, and can exchange data. The Bluetooth protocol stack includes multiple Bluetooth threads, one of which is a Bluetooth audio processing thread. The Bluetooth audio processing thread is used to process audio tasks and is distinguished from other types of Bluetooth tasks, thereby avoiding the situation of task queuing in the Bluetooth thread and solving the communication delay problem when processing a single-threaded Bluetooth protocol stack, thereby improving the responsiveness of audio control.

[0166] The display unit is used to display audio related information, that is, audio playback information;

[0167] The communication link of the audio control method provided in the embodiment of the present application may include but is not limited to the following steps:

[0168] The user presses a button on the square control system;

[0169] The square control system responds to the user's operation, generates key operations, and sends them to the vehicle system via LIN signals;

[0170] After receiving the key operation transmitted by the square control system, the vehicle-mounted system converts the key operation into a key signal, and performs a key value query on the key signal based on the key mapping relationship table of the square control system to obtain the key value data;

[0171] The vehicle-mounted system performs task query based on the key value data of the preset key task mapping table to obtain the target task type;

[0172] Wherein, the target task types include audio tasks and non-audio tasks. If the key operation is a non-audio task, the key operation is processed in other Bluetooth threads in the Bluetooth protocol stack except the Bluetooth audio processing thread according to the conventional key operation;

[0173] If the key operation is an audio task, the audio key operation is processed through the Bluetooth audio processing thread in the Bluetooth protocol stack;

[0174] If the key operation is an audio task, filter out the audio control operation from the key operation according to the target task type, and send the audio control operation to the middle layer;

[0175] The middle layer sends the audio control operation to the Bluetooth audio processing thread in the Bluetooth protocol stack;

[0176] The Bluetooth audio processing thread sends the audio control operation to the mobile terminal.

[0177] The mobile terminal receives the audio control operation sent by the vehicle-mounted system, and sends a received command feedback message to the Bluetooth audio processing thread of the vehicle-mounted system;

[0178] The mobile terminal switches the audio according to the audio control operation and feeds back the audio switching information to the vehicle-mounted system;

[0179] The Bluetooth audio processing thread of the vehicle-mounted system receives the audio switching information sent by the mobile terminal, and obtains instructions according to the audio information sent by the audio switching information;

[0180] The mobile terminal receives the audio information acquisition instruction sent by the vehicle system according to the audio switching information, acquires information based on the audio information acquisition instruction, obtains the target audio information; and sends the target audio information to the Bluetooth audio processing thread;

[0181] The vehicle-mounted system receives the target audio information and performs audio control on the audio playback information on the display unit according to the target audio information; specifically, the target audio information includes audio data and audio attributes of the audio data; the playback audio is controlled according to the audio data; and the audio information of the playback content is updated based on the audio attributes.

[0182] It should be noted that in the process of interaction between the Bluetooth protocol stack and the mobile terminal, if the Bluetooth protocol stack is single-threaded, it can only process one task independently, and tasks need to be queued and processed in sequence or according to preset priorities, which may cause some tasks to be delayed and have poor responsiveness. In the relevant technology, audio tasks have a lower task priority, so when controlling Bluetooth audio, low responsiveness often occurs.

[0183] Based on this, multiple Bluetooth threads are set up in the Bluetooth protocol stack, one of which is the Bluetooth audio processing thread. The Bluetooth audio processing thread is used to process audio tasks and is distinguished from other types of Bluetooth tasks, thereby avoiding task queuing in the Bluetooth thread and solving the communication delay problem during single-threaded Bluetooth protocol stack processing, thereby improving the responsiveness of audio control.

[0184] In another specific embodiment, the vehicle-mounted system further includes an application layer and a processing unit;

[0185] Wherein, the processing unit is used to process the key operation transmitted by the opposite control system;

[0186] The application layer is the audio playback application of the vehicle system. The specific information of the application layer will be displayed on the display unit, that is, the audio playback information.

[0187] The complete communication link of the audio control method provided in the embodiment of the present application may include but is not limited to the following steps:

[0188] The user presses a button on the square control system;

[0189] The square control system responds to the user's operation, generates key operations, and sends them to the vehicle system via LIN signals;

[0190] After receiving the key operation transmitted by the square control system, the processing unit of the vehicle system converts the key operation into a key signal, and performs a key value query on the key signal based on the key mapping relationship table of the square control system to obtain the key value data;

[0191] The processing unit transparently transmits the key value data to the middle layer;

[0192] The middle layer sends the key value data to the application layer so that the application layer can identify the task type;

[0193] After receiving the key value data, the application layer performs task query based on the key value data in the preset key task mapping table to obtain the target task type;

[0194] Wherein, the target task types include audio tasks and non-audio tasks. If the key operation is a non-audio task, the key operation is processed in other Bluetooth threads in the Bluetooth protocol stack except the Bluetooth audio processing thread according to the conventional key operation;

[0195] If the key operation is an audio task, the audio key operation is processed through the Bluetooth audio processing thread in the Bluetooth protocol stack;

[0196] If the key operation is an audio task, the application layer filters out the audio control operation from the key operation according to the target task type, and sends the audio control operation to the middle layer;

[0197] The middle layer sends the audio control operation to the Bluetooth audio processing thread in the Bluetooth protocol stack;

[0198] The Bluetooth audio processing thread sends the audio control operation to the mobile terminal.

[0199] The mobile terminal receives the audio control operation sent by the vehicle-mounted system, and sends a received command feedback message to the Bluetooth audio processing thread of the vehicle-mounted system;

[0200] The mobile terminal switches the audio according to the audio control operation and feeds back the audio switching information to the vehicle-mounted system;

[0201] The Bluetooth audio processing thread of the vehicle-mounted system receives the audio switching information sent by the mobile terminal, and obtains instructions according to the audio information sent by the audio switching information;

[0202] The mobile terminal receives the audio information acquisition instruction sent by the vehicle system according to the audio switching information, acquires information based on the audio information acquisition instruction, obtains the target audio information; and sends the target audio information to the Bluetooth audio processing thread;

[0203] The Bluetooth audio processing thread sends the target audio information to the middle layer;

[0204] The middle layer sends the target audio information to the application layer;

[0205] The application layer receives the target audio information and performs audio control on the audio playback information on the display unit according to the target audio information; specifically, the target audio information includes audio data and audio attributes of the audio data; the playback audio is controlled according to the audio data; and the audio information of the playback content is updated based on the audio attributes.

[0206] It should be noted that by setting up an intermediate layer, an application layer, a Bluetooth protocol stack, a processing unit, and a display unit in the vehicle system, the various functional modules can be better separated and the scalability and stability of the system can be improved.

[0207] In addition, it can fully reflect the tasks handled by each module in the vehicle system and reflect the complete communication link, so as to optimize the processing time of each link in the communication link and improve the responsiveness of audio adjustment.

[0208] In a specific usage scenario, there are two tasks. The first task is to synchronize the address book information in the mobile terminal with the vehicle system, and the second task is to switch Bluetooth music.

[0209] Assuming that the Bluetooth protocol stack is single-threaded and the Bluetooth protocol stack is processing the first task, the second task needs to enter the queue and wait for the first task to complete before executing the second task. During the waiting process, it takes several seconds or even longer to wait, and music switching is a highly perceived behavior. If there is a long waiting time between the user's switching of Bluetooth music and the completion of the Bluetooth music switching action by the vehicle system, the user can perceive a significant delay, and the entire switching process is "unresponsive", which will greatly reduce the user experience.

[0210] And suppose the Bluetooth protocol stack is multi-threaded, and one of the threads is specifically used to process audio tasks. Even if the Bluetooth protocol stack is processing the first task, there is still a thread in the Bluetooth protocol stack that can handle the request to switch Bluetooth music. The vehicle system can quickly respond to the user's request to switch music, shortening the waiting time between the user's switching Bluetooth music and the vehicle system's completion of the Bluetooth music switching action, improving the responsiveness of audio control, and the user cannot perceive obvious delays. The entire switching process is "hand-following", which can enhance the user experience.

[0211] See also Fig.10 The present application embodiment provides an audio control device, which is applied to a vehicle-mounted system and can realize the above Figures 2 to 7 In any of the audio control methods shown, the device comprises:

[0212] The key operation receiving module 901 is used to receive the key operation transmitted by the party control system;

[0213] The task type identification module 902 is used to identify the task type of the key operation and obtain the target task type;

[0214] A control operation screening module 903, used to screen audio control operations from key operations according to target task types;

[0215] A control operation sending module 904, used to send the audio control operation to the mobile terminal;

[0216] The control information receiving module 905 is used to receive the target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data;

[0217] The audio control module 906 is used to perform audio control on preset audio playing information according to audio data and audio attributes; wherein the audio playing information includes at least one of the following: playing audio, playing content.

[0218] In the task type identification module 902 of some embodiments, the task type identification module 902 further includes:

[0219] A signal conversion unit, used for converting key operations into key signals;

[0220] A key value conversion unit, used for converting the key signal into a key value to obtain key value data;

[0221] The task classification unit is used to classify the key-value data into tasks to obtain the target task type.

[0222] In some embodiments of the key-value conversion unit, the key-value conversion unit further includes:

[0223] A mapping relationship table acquisition subunit is used to acquire a key mapping relationship table from a party control system;

[0224] The key value query subunit is used to perform key value query on the key signal based on the key mapping relationship table to obtain key value data.

[0225] In some embodiments, the task classification unit further includes:

[0226] The task mapping table acquisition subunit is used to acquire a preset key task mapping table; wherein the key task mapping table stores the task type corresponding to each key value data;

[0227] The task query subunit is used to perform task query on the key value data based on the key task mapping table to obtain the target task type.

[0228] In the control operation sending module 904 of some embodiments, the control operation sending module 904 further includes:

[0229] A control operation storage unit, used to store audio control operations in a Bluetooth audio processing thread;

[0230] The control operation distribution unit is used to send the audio control operation to the mobile terminal through the Bluetooth audio processing thread, wherein the audio control operation is used to enable the mobile terminal to generate target audio information.

[0231] In some embodiments of the audio control module 906, the audio control module 906 further includes:

[0232] An audio playback control unit, used to control the playback of the audio according to the audio data;

[0233] The audio information updating unit is used to update the audio information of the playback content based on the audio attribute.

[0234] The specific implementation of the audio control device is the same as the above Figures 2 to 7 The specific embodiments of any of the audio control methods shown are basically the same and will not be described in detail herein.

[0235] See also Fig.11 Another embodiment of the present application provides an audio control device, which is applied to a mobile terminal and can realize the above Figures 8 to 9 In any of the audio control methods shown, the device comprises:

[0236] The control operation receiving module 1001 is used to receive the audio control operation sent by the vehicle system; wherein the audio control operation is Figures 2 to 7 Any of the audio control methods shown is obtained;

[0237] An audio switching module 1002 is used to perform audio switching based on an audio control operation to obtain target audio information;

[0238] The audio information sending module 1003 is used to send the target audio information to the vehicle-mounted system.

[0239] In some embodiments of the audio switching module 1002, the audio switching module 1002 further includes:

[0240] An audio switching control unit, used to perform audio switching according to audio control operations and to feed back audio switching information to the vehicle-mounted system;

[0241] An instruction receiving unit, used for receiving an audio information acquisition instruction sent by the vehicle-mounted system according to the audio switching information;

[0242] The information acquisition unit is used to acquire information based on the audio information acquisition instruction to obtain target audio information.

[0243] The specific implementation of the audio control device is the same as the above Figures 8 to 9 The specific embodiments of any of the audio control methods shown are basically the same and will not be described in detail herein.

[0244] The embodiment of the present application also provides a vehicle, the vehicle includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to achieve the above Figures 2 to 7Any of the audio control methods shown. The vehicle can be any intelligent terminal including an onboard computer.

[0245] See also Fig.12 , Fig.12 The hardware structure of a vehicle of another embodiment is illustrated, and the vehicle includes:

[0246] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0247] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1102 and are called and executed by the processor 1101 in the embodiments of this application. Figures 2 to 7 Any of the audio control methods shown.

[0248] Input / output interface 1103, used to implement information input and output;

[0249] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0250] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0251] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0252] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned Figures 2 to 7 Any of the audio control methods shown, or, as Figures 8 to 9 Any of the audio control methods shown.

[0253] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0254] In some embodiments, the vehicle can be a gasoline vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.

[0255] The audio control method, device, vehicle and storage medium provided in the embodiments of the present application receive the key operation transmitted by the control system; identify the task type of the key operation to obtain the target task type; filter out the audio control operation from the key operation according to the target task type; send the audio control operation to the mobile terminal; receive the target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data; perform audio control on the preset audio playback information according to the audio data and the audio attributes. The embodiments of the present application identify the task type of the key operation output by the control system, filter out the audio control operation, and can respond to the audio control operation alone without waiting for other tasks to be completed before executing the audio control operation, thereby improving the responsiveness of the audio control.

[0256] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0257] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0258] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0259] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0260] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0261] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0262] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0263] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0264] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0265] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0266] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An audio control method, characterized in that: Applied to a vehicle-mounted system, the vehicle-mounted system is used for connecting a mobile terminal to a Bluetooth device, the vehicle-mounted system includes a Bluetooth protocol stack, the Bluetooth protocol stack includes multiple Bluetooth threads, one of the Bluetooth threads is a Bluetooth audio processing thread, and the method includes: The receiving party controls the key operations transmitted by the system; Performing task type identification on the key operation to obtain a target task type; Filtering out an audio control operation from the key operations according to the target task type; wherein the audio control operation is an audio playback control; Storing the audio control operation in the Bluetooth audio processing thread; Sending the audio control operation to the mobile terminal through the Bluetooth audio processing thread, wherein the audio control operation is used to enable the mobile terminal to generate target audio information; Receiving the target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data, and the audio data includes music data, podcast data, and video audio data; Performing audio control on preset audio playback information according to the audio data and the audio attribute; wherein the audio playback information includes at least one of the following: playing audio, playing content; The step of performing audio control on the preset audio playback information according to the audio data and the audio attribute includes: Controlling the playback of the audio according to the audio data; The audio information of the playback content is updated based on the audio attribute.

2. The method according to claim 1, characterized in that The step of performing task type identification on the key operation to obtain a target task type includes: Converting the key operation into a key signal; Performing key value conversion on the key signal to obtain key value data; The key-value data is task classified to obtain the target task type.

3. The method according to claim 2, characterized in that The step of converting the key signal into a key value to obtain key value data includes: Acquire a key mapping relationship table from the party control system; A key value query is performed on the key signal based on the key mapping relationship table to obtain the key value data.

4. The method according to claim 2, characterized in that: The step of performing task classification on the key value data to obtain the target task type includes: Obtaining a preset key task mapping table; wherein the key task mapping table stores the task type corresponding to each key value data; A task query is performed on the button key value data based on the button task mapping table to obtain the target task type.

5. An audio control method, characterized in that: Applied to a mobile terminal, the method comprises: Receiving an audio control operation sent by an in-vehicle system; wherein the audio control operation is obtained by the method according to any one of claims 1 to 4; Perform audio switching based on the audio control operation to obtain target audio information; The target audio information is sent to the vehicle-mounted system.

6. The method according to claim 5, characterized in that The performing audio switching based on the audio control operation to obtain target audio information includes: Performing audio switching according to the audio control operation, and feeding back audio switching information to the in-vehicle system; receiving an audio information acquisition instruction sent by the vehicle-mounted system according to the audio switching information; Information acquisition is performed based on the audio information acquisition instruction to obtain the target audio information.

7. An audio control device, characterized in that: Applied to a vehicle-mounted system, the vehicle-mounted system is used for connecting a mobile terminal to a Bluetooth device, the vehicle-mounted system includes a Bluetooth protocol stack, the Bluetooth protocol stack includes a plurality of Bluetooth threads, one of the Bluetooth threads is a Bluetooth audio processing thread, and the device includes: A key operation receiving module is used to receive key operations transmitted by the party control system; A task type identification module, used to identify the task type of the key operation to obtain a target task type; A control operation screening module, used for screening out audio control operations from the key operations according to the target task type; wherein the audio control operation is audio playback control; A control operation sending module, used for storing the audio control operation in the Bluetooth audio processing thread; sending the audio control operation to the mobile terminal through the Bluetooth audio processing thread, wherein the audio control operation is used to enable the mobile terminal to generate target audio information; A control information receiving module, configured to receive the target audio information sent by the mobile terminal according to the audio control operation; wherein the target audio information includes audio data and audio attributes of the audio data, and the audio data includes music data, podcast data, and video audio data; An audio control module, used for performing audio control on preset audio playing information according to the audio data and the audio attribute; wherein the audio playing information includes at least one of the following: playing audio, playing content; The step of performing audio control on the preset audio playback information according to the audio data and the audio attribute includes: Controlling the playback of the audio according to the audio data; The audio information of the playback content is updated based on the audio attribute.

8. An audio control device, characterized in that: Applied to a mobile terminal, the device comprises: A control operation receiving module, used for receiving an audio control operation sent by the vehicle-mounted system; wherein the audio control operation is obtained by the method according to any one of claims 1 to 4; An audio switching module, used to perform audio switching based on the audio control operation to obtain target audio information; The audio information sending module is used to send the target audio information to the vehicle-mounted system.

9. A vehicle, characterized in that: The vehicle comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 4 or the method described in any one of claims 5 to 6 is implemented.